Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-11-18T14:57:50Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T14:57:50Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskyLAIFs/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskyLAIFs/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/security.properties
2026-11-18 14:57:57: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskyLAIFs/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-taskyLAIFs/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20261118 14:58:05.308992 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42447
--webserver_interface=127.0.72.190
--webserver_port=0
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 14:58:05.649632 572 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:05.649937 572 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:05.650048 572 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:05.658579 572 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:05.658667 572 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:05.658735 572 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:05.658797 572 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:05.669039 572 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:05.670749 572 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:05.672564 572 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:05.680768 578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:05.680800 577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:05.682240 580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:05.683853 572 server_base.cc:1061] running on GCE node
I20261118 14:58:05.684612 572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:05.686033 572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:05.687237 572 hybrid_clock.cc:648] HybridClock initialized: now 1795013885687143 us; error 86 us; skew 500 ppm
I20261118 14:58:05.687547 572 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:05.689654 572 webserver.cc:577] Webserver started at http://127.0.72.190:44205/ using document root <none> and password file <none>
I20261118 14:58:05.690414 572 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:05.690517 572 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:05.690886 572 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 14:58:05.693295 572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/instance:
uuid: "0ed65a3719634e519d783540afd6189e"
format_stamp: "Formatted at 2026-11-18 14:58:05 on dist-test-slave-qxzn"
I20261118 14:58:05.694036 572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal/instance:
uuid: "0ed65a3719634e519d783540afd6189e"
format_stamp: "Formatted at 2026-11-18 14:58:05 on dist-test-slave-qxzn"
I20261118 14:58:05.699656 572 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20261118 14:58:05.702867 586 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:05.704304 572 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20261118 14:58:05.704447 572 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
uuid: "0ed65a3719634e519d783540afd6189e"
format_stamp: "Formatted at 2026-11-18 14:58:05 on dist-test-slave-qxzn"
I20261118 14:58:05.704632 572 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:05.718137 572 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:05.719051 572 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:05.719405 572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:05.739753 572 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42447
I20261118 14:58:05.739771 637 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42447 every 8 connection(s)
I20261118 14:58:05.741272 572 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
I20261118 14:58:05.744776 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 572
I20261118 14:58:05.745019 290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal/instance
I20261118 14:58:05.746042 638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:05.747498 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:46707
--webserver_interface=127.0.72.189
--webserver_port=0
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 14:58:05.752971 638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } has no permanent_uuid. Determining permanent_uuid...
I20261118 14:58:05.775213 638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } has no permanent_uuid. Determining permanent_uuid...
W20261118 14:58:05.776314 588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:05.778096 638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:46707: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:05.827407 638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } attempt: 1
W20261118 14:58:05.829314 638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:46707: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:05.897552 638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } attempt: 2
W20261118 14:58:05.899454 638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:46707: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:05.990746 638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } attempt: 3
W20261118 14:58:05.992995 638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:46707: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:06.091070 639 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:06.091346 639 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:06.091419 639 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:06.099720 639 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:06.099802 639 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:06.099862 639 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:06.099908 639 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:06.109872 639 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:46707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:06.111395 639 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:06.113134 639 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:06.121275 648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:06.121723 647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:06.122315 650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:06.122592 639 server_base.cc:1061] running on GCE node
I20261118 14:58:06.123236 639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:06.124492 639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:06.125687 639 hybrid_clock.cc:648] HybridClock initialized: now 1795013886125636 us; error 43 us; skew 500 ppm
I20261118 14:58:06.125988 639 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:06.127909 639 webserver.cc:577] Webserver started at http://127.0.72.189:42561/ using document root <none> and password file <none>
I20261118 14:58:06.128544 639 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:06.128638 639 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:06.128968 639 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 14:58:06.131147 639 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/instance:
uuid: "7c9ef96ff074450a879e66787486386d"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:06.131860 639 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal/instance:
uuid: "7c9ef96ff074450a879e66787486386d"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:06.136220 638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } attempt: 4
I20261118 14:58:06.136777 639 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
W20261118 14:58:06.137951 638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:46707: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:06.140142 656 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:06.141428 639 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:06.141564 639 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
uuid: "7c9ef96ff074450a879e66787486386d"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:06.141739 639 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:06.168368 639 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:06.169219 639 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:06.169457 639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:06.189335 639 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:46707
I20261118 14:58:06.189342 707 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:46707 every 8 connection(s)
I20261118 14:58:06.190843 639 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
I20261118 14:58:06.194036 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 639
I20261118 14:58:06.194232 290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal/instance
I20261118 14:58:06.195443 708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:06.195585 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:33967
--webserver_interface=127.0.72.188
--webserver_port=0
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 14:58:06.202786 708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } has no permanent_uuid. Determining permanent_uuid...
I20261118 14:58:06.215229 708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } has no permanent_uuid. Determining permanent_uuid...
I20261118 14:58:06.223857 708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
W20261118 14:58:06.224880 659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
W20261118 14:58:06.226699 708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:06.276029 708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } attempt: 1
W20261118 14:58:06.277915 708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:06.346105 708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } attempt: 2
W20261118 14:58:06.348906 708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:06.437177 638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } attempt: 5
I20261118 14:58:06.440104 708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } attempt: 3
W20261118 14:58:06.441926 708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:06.443859 638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
W20261118 14:58:06.445479 638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:06.496758 638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } attempt: 1
W20261118 14:58:06.498704 638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
W20261118 14:58:06.531138 709 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:06.531407 709 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:06.531479 709 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:06.539732 709 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:06.539819 709 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:06.539870 709 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:06.539916 709 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:06.549917 709 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.72.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:06.551551 709 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:06.553206 709 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:06.560951 718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:06.561012 717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:06.561278 720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:06.562290 709 server_base.cc:1061] running on GCE node
I20261118 14:58:06.562963 709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:06.564224 709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:06.565404 709 hybrid_clock.cc:648] HybridClock initialized: now 1795013886565365 us; error 34 us; skew 500 ppm
I20261118 14:58:06.565682 709 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:06.566933 638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } attempt: 2
I20261118 14:58:06.568070 709 webserver.cc:577] Webserver started at http://127.0.72.188:42503/ using document root <none> and password file <none>
W20261118 14:58:06.568670 638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:06.568763 709 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:06.568854 709 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:06.569164 709 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 14:58:06.571383 709 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/instance:
uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:06.572049 709 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal/instance:
uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:06.576867 709 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20261118 14:58:06.579980 726 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:06.581418 709 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20261118 14:58:06.581552 709 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:06.581733 709 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:06.585151 708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } attempt: 4
W20261118 14:58:06.587162 708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:06.622033 709 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:06.622993 709 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:06.623242 709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:06.642310 709 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.188:33967
I20261118 14:58:06.642334 777 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.188:33967 every 8 connection(s)
I20261118 14:58:06.643733 709 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
I20261118 14:58:06.647867 778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:06.651980 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 709
I20261118 14:58:06.652138 290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal/instance
I20261118 14:58:06.653683 778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } has no permanent_uuid. Determining permanent_uuid...
I20261118 14:58:06.666978 778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } has no permanent_uuid. Determining permanent_uuid...
I20261118 14:58:06.674103 778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
I20261118 14:58:06.674870 638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } attempt: 3
I20261118 14:58:06.692243 778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap starting.
I20261118 14:58:06.695432 778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:06.696805 778 log.cc:826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:06.699420 778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: No bootstrap required, opened a new log
I20261118 14:58:06.699493 638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Bootstrap starting.
I20261118 14:58:06.703646 638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:06.704906 638 log.cc:826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:06.706297 778 raft_consensus.cc:360] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.706641 778 raft_consensus.cc:386] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:06.706770 778 raft_consensus.cc:748] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b90720bc0b34e4994abf4fc6f9e59e1, State: Initialized, Role: FOLLOWER
I20261118 14:58:06.707469 778 consensus_queue.cc:261] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.707713 638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: No bootstrap required, opened a new log
I20261118 14:58:06.708998 788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:06.709334 788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:06.710220 778 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:06.715219 638 raft_consensus.cc:360] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.715646 638 raft_consensus.cc:386] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:06.715811 638 raft_consensus.cc:748] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed65a3719634e519d783540afd6189e, State: Initialized, Role: FOLLOWER
I20261118 14:58:06.716696 638 consensus_queue.cc:261] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.717584 801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:06.717828 801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:06.718995 638 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: configured and running, proceeding with master startup.
W20261118 14:58:06.722134 592 tablet.cc:2441] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 14:58:06.725920 803 catalog_manager.cc:820] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20261118 14:58:06.729671 709 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: OK, instance_id { permanent_uuid: "0ed65a3719634e519d783540afd6189e" instance_seqno: 1795013885733973 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20261118 14:58:06.736438 709 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:06.783696 788 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:06.783985 788 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.785806 788 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
I20261118 14:58:06.787120 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e" is_pre_election: true
I20261118 14:58:06.787750 613 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b90720bc0b34e4994abf4fc6f9e59e1 in term 0.
I20261118 14:58:06.788646 727 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters:
I20261118 14:58:06.789188 788 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:06.789335 788 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:06.789414 788 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:06.792670 788 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.793396 788 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 1 election: Requested vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
I20261118 14:58:06.794020 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e"
I20261118 14:58:06.794234 613 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:06.797672 613 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b90720bc0b34e4994abf4fc6f9e59e1 in term 1.
I20261118 14:58:06.798172 727 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters:
I20261118 14:58:06.798609 788 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:06.799285 788 raft_consensus.cc:705] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 1 LEADER]: Becoming Leader. State: Replica: 9b90720bc0b34e4994abf4fc6f9e59e1, State: Running, Role: LEADER
I20261118 14:58:06.799898 788 consensus_queue.cc:238] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.803911 815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b90720bc0b34e4994abf4fc6f9e59e1. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:06.804236 815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:06.812366 817 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 14:58:06.815085 819 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:06.820705 817 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:06.829317 801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b90720bc0b34e4994abf4fc6f9e59e1. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:06.829584 801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:06.833590 815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:06.833789 815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:06.850251 801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:06.850502 801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:06.886390 708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } attempt: 5
I20261118 14:58:06.900472 708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap starting.
I20261118 14:58:06.900977 817 catalog_manager.cc:1398] Generated new cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:06.901078 817 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:06.906111 708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:06.907490 708 log.cc:826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:06.917387 708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: No bootstrap required, opened a new log
W20261118 14:58:06.922468 828 catalog_manager.cc:1609] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20261118 14:58:06.922904 828 catalog_manager.cc:898] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20261118 14:58:06.903255 572 master_runner.cc:187] Error getting master registration for 127.0.72.189:46707: OK, instance_id { permanent_uuid: "7c9ef96ff074450a879e66787486386d" instance_seqno: 1795013886184083 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20261118 14:58:06.924345 572 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:06.924861 708 raft_consensus.cc:360] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.925256 708 raft_consensus.cc:386] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:06.925441 708 raft_consensus.cc:748] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9ef96ff074450a879e66787486386d, State: Initialized, Role: FOLLOWER
I20261118 14:58:06.926277 708 consensus_queue.cc:261] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:06.942977 833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:06.943385 833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:06.944169 708 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:06.944657 681 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:06.944734 683 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d" is_pre_election: true
I20261118 14:58:06.945216 682 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d"
I20261118 14:58:06.950484 819 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:06.955610 817 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 14:58:06.957335 817 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:06.989048 817 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Generated new TSK 0
I20261118 14:58:06.991036 817 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:06.992563 817 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:06.993834 817 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 14:58:07.003756 788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:07.004055 788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:07.003976 833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b90720bc0b34e4994abf4fc6f9e59e1. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:07.004202 833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:07.004776 841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:07.005074 841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:07.005802 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.72.129:0
--local_ip_for_outbound_sockets=127.0.72.129
--webserver_interface=127.0.72.129
--webserver_port=0
--tserver_master_addrs=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 14:58:07.082058 855 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:07.082172 855 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: loading cluster ID for follower catalog manager: success
I20261118 14:58:07.086311 855 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: acquiring CA information for follower catalog manager: success
I20261118 14:58:07.089078 855 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:07.510991 857 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:07.511274 857 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:07.513602 857 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:07.514919 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e" is_pre_election: true
I20261118 14:58:07.515707 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:07.517310 657 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c9ef96ff074450a879e66787486386d; no voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:07.517791 857 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 14:58:07.555701 848 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:07.556139 848 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:07.556298 848 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:07.570087 848 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:07.570390 848 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.72.129
I20261118 14:58:07.587306 848 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.72.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.72.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.72.129
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:07.589597 848 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:07.592128 848 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:07.604859 862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:07.605495 863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:07.606191 865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:07.607380 848 server_base.cc:1061] running on GCE node
I20261118 14:58:07.607942 848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:07.608950 848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:07.610157 848 hybrid_clock.cc:648] HybridClock initialized: now 1795013887610075 us; error 75 us; skew 500 ppm
I20261118 14:58:07.610496 848 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:07.612696 848 webserver.cc:577] Webserver started at http://127.0.72.129:38325/ using document root <none> and password file <none>
I20261118 14:58:07.613400 848 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:07.613512 848 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:07.613911 848 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 14:58:07.616456 848 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/data/instance:
uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
format_stamp: "Formatted at 2026-11-18 14:58:07 on dist-test-slave-qxzn"
I20261118 14:58:07.617206 848 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/instance:
uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
format_stamp: "Formatted at 2026-11-18 14:58:07 on dist-test-slave-qxzn"
I20261118 14:58:07.622745 848 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20261118 14:58:07.626185 871 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:07.627885 848 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20261118 14:58:07.628039 848 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal
uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
format_stamp: "Formatted at 2026-11-18 14:58:07 on dist-test-slave-qxzn"
I20261118 14:58:07.628239 848 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:07.649902 848 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:07.650935 848 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:07.651186 848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:07.652279 848 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 14:58:07.655045 848 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 14:58:07.655180 848 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:07.655308 848 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 14:58:07.655385 848 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:07.697161 848 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.129:41463
I20261118 14:58:07.697278 983 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.129:41463 every 8 connection(s)
I20261118 14:58:07.699561 848 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/data/info.pb
I20261118 14:58:07.701925 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 848
I20261118 14:58:07.702162 290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/instance
I20261118 14:58:07.710057 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.72.130:0
--local_ip_for_outbound_sockets=127.0.72.130
--webserver_interface=127.0.72.130
--webserver_port=0
--tserver_master_addrs=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 14:58:07.744194 985 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
I20261118 14:58:07.744218 984 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:07.744629 984 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:07.745185 985 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:07.745841 985 heartbeater.cc:508] Master 127.0.72.190:42447 requested a full tablet report, sending...
I20261118 14:58:07.745901 984 heartbeater.cc:508] Master 127.0.72.188:33967 requested a full tablet report, sending...
I20261118 14:58:07.745859 986 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:07.746409 986 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:07.746912 986 heartbeater.cc:508] Master 127.0.72.189:46707 requested a full tablet report, sending...
I20261118 14:58:07.748510 602 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:07.748518 673 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:07.750830 743 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:07.753007 743 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.0.72.129:47561
I20261118 14:58:07.902757 995 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:07.902987 995 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:07.904407 995 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:07.904973 681 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d" is_pre_election: true
I20261118 14:58:07.905068 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:07.905743 588 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:07.906133 995 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:07.925479 828 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:07.925576 828 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: loading cluster ID for follower catalog manager: success
I20261118 14:58:07.929451 828 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: acquiring CA information for follower catalog manager: success
I20261118 14:58:07.932379 828 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20261118 14:58:08.087522 994 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:08.087805 994 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:08.087930 994 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:08.096277 994 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:08.096487 994 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.72.130
I20261118 14:58:08.106443 994 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.72.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.72.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.72.130
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:08.108101 994 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:08.109874 994 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:08.119589 1003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:08.119491 1000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:08.119529 1001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:08.120236 994 server_base.cc:1061] running on GCE node
I20261118 14:58:08.121145 994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:08.122064 994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:08.123243 994 hybrid_clock.cc:648] HybridClock initialized: now 1795013888123175 us; error 59 us; skew 500 ppm
I20261118 14:58:08.123509 994 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:08.125437 994 webserver.cc:577] Webserver started at http://127.0.72.130:45323/ using document root <none> and password file <none>
I20261118 14:58:08.126102 994 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:08.126196 994 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:08.126579 994 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 14:58:08.128762 994 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/data/instance:
uuid: "245c4fe5e44448d98db1eed6329d150d"
format_stamp: "Formatted at 2026-11-18 14:58:08 on dist-test-slave-qxzn"
I20261118 14:58:08.129460 994 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/instance:
uuid: "245c4fe5e44448d98db1eed6329d150d"
format_stamp: "Formatted at 2026-11-18 14:58:08 on dist-test-slave-qxzn"
I20261118 14:58:08.134490 994 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20261118 14:58:08.137488 1009 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:08.138827 994 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20261118 14:58:08.138967 994 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal
uuid: "245c4fe5e44448d98db1eed6329d150d"
format_stamp: "Formatted at 2026-11-18 14:58:08 on dist-test-slave-qxzn"
I20261118 14:58:08.139133 994 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:08.171859 994 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:08.172714 994 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:08.172945 994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:08.173875 994 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 14:58:08.175552 994 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 14:58:08.175644 994 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:08.176141 994 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 14:58:08.176220 994 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:08.209079 994 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.130:38257
I20261118 14:58:08.209136 1121 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.130:38257 every 8 connection(s)
I20261118 14:58:08.210474 994 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/data/info.pb
I20261118 14:58:08.212980 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 994
I20261118 14:58:08.213161 290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/instance
I20261118 14:58:08.218667 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.72.131:0
--local_ip_for_outbound_sockets=127.0.72.131
--webserver_interface=127.0.72.131
--webserver_port=0
--tserver_master_addrs=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 14:58:08.235504 1124 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:08.235528 1122 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:08.235867 1124 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:08.235873 1122 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:08.236733 1124 heartbeater.cc:508] Master 127.0.72.189:46707 requested a full tablet report, sending...
I20261118 14:58:08.236759 1122 heartbeater.cc:508] Master 127.0.72.188:33967 requested a full tablet report, sending...
I20261118 14:58:08.237478 1123 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
I20261118 14:58:08.237619 1123 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:08.238328 743 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:08.238538 1123 heartbeater.cc:508] Master 127.0.72.190:42447 requested a full tablet report, sending...
I20261118 14:58:08.238832 673 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:08.239383 743 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.0.72.130:50135
I20261118 14:58:08.239445 602 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:08.256240 984 heartbeater.cc:500] Master 127.0.72.188:33967 was elected leader, sending a full tablet report...
I20261118 14:58:08.526916 1130 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:08.527149 1130 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:08.527803 1130 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:08.528383 681 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d" is_pre_election: true
I20261118 14:58:08.528427 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:08.529071 589 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:08.529346 1130 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 14:58:08.556370 1129 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:08.556739 1129 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:08.556870 1129 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:08.565907 1129 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:08.566147 1129 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.72.131
I20261118 14:58:08.576366 1129 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.72.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.72.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.72.131
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:08.578207 1129 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:08.580803 1129 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:08.591401 1138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:08.591480 1135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:08.591432 1136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:08.592505 1129 server_base.cc:1061] running on GCE node
I20261118 14:58:08.593096 1129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:08.594930 1129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:08.596186 1129 hybrid_clock.cc:648] HybridClock initialized: now 1795013888596118 us; error 50 us; skew 500 ppm
I20261118 14:58:08.596516 1129 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:08.598728 1129 webserver.cc:577] Webserver started at http://127.0.72.131:42971/ using document root <none> and password file <none>
I20261118 14:58:08.599445 1129 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:08.599545 1129 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:08.599903 1129 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 14:58:08.602254 1129 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/data/instance:
uuid: "c01f1b88968f4905ba5a401a81ade9f7"
format_stamp: "Formatted at 2026-11-18 14:58:08 on dist-test-slave-qxzn"
I20261118 14:58:08.603011 1129 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/instance:
uuid: "c01f1b88968f4905ba5a401a81ade9f7"
format_stamp: "Formatted at 2026-11-18 14:58:08 on dist-test-slave-qxzn"
I20261118 14:58:08.608021 1129 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20261118 14:58:08.611222 1144 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:08.612608 1129 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20261118 14:58:08.612756 1129 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal
uuid: "c01f1b88968f4905ba5a401a81ade9f7"
format_stamp: "Formatted at 2026-11-18 14:58:08 on dist-test-slave-qxzn"
I20261118 14:58:08.612926 1129 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:08.639786 1129 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:08.640918 1129 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:08.641160 1129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:08.642076 1129 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 14:58:08.643764 1129 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 14:58:08.643852 1129 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:08.643950 1129 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 14:58:08.644018 1129 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:08.677039 1129 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.131:45343
I20261118 14:58:08.677121 1256 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.131:45343 every 8 connection(s)
I20261118 14:58:08.678514 1129 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/data/info.pb
I20261118 14:58:08.679373 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 1129
I20261118 14:58:08.679581 290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/instance
I20261118 14:58:08.698632 1257 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:08.698654 1258 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
I20261118 14:58:08.699046 1257 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:08.699047 1258 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:08.700289 1258 heartbeater.cc:508] Master 127.0.72.190:42447 requested a full tablet report, sending...
I20261118 14:58:08.700320 1257 heartbeater.cc:508] Master 127.0.72.188:33967 requested a full tablet report, sending...
I20261118 14:58:08.701963 743 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:08.702224 602 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:08.703014 743 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.0.72.131:57667
I20261118 14:58:08.703480 1259 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:08.703616 1259 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:08.704138 1259 heartbeater.cc:508] Master 127.0.72.189:46707 requested a full tablet report, sending...
I20261118 14:58:08.705112 673 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:08.707455 290 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20261118 14:58:08.740330 743 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_ab0fd883cbec4e0b8af7febddffe78c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.741344 742 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_b1a3822c61cb4043b4b9ddeaf29cc405"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.741732 1122 heartbeater.cc:500] Master 127.0.72.188:33967 was elected leader, sending a full tablet report...
I20261118 14:58:08.746109 740 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_59239feaba874d6890fe8e86d57545e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.747479 738 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_41739c72be484e8fb242c925dd0e2c32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.750144 739 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_38deaf819a5e430090d96b02cbd9cdc3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.750921 737 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_d908b160aac24b678c5aff465735ea9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.751775 736 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.752189 735 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_ced554c65cd4465d9ddab052dc32efcf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.753172 734 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_6c4a501fa569491080a3b4988350b5b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:08.753765 741 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_7886615812a14a0492f39f2694999785"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20261118 14:58:08.758517 743 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ab0fd883cbec4e0b8af7febddffe78c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:08.759577 742 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_b1a3822c61cb4043b4b9ddeaf29cc405 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:08.769433 290 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu pid 639
I20261118 14:58:08.770996 703 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 14:58:08.932773 290 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu with pid 639
W20261118 14:58:08.962682 728 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:08.962899 728 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:08.969298 728 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 -> Peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Couldn't send request to peer 7c9ef96ff074450a879e66787486386d. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:09.056741 1307 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:09.057134 1307 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:09.065392 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:09.068600 1307 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:09.096585 588 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:09.096832 588 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:09.097206 1307 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:09.169727 742 catalog_manager.cc:2215] created HMS catalog entry for table [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:09.205421 1257 heartbeater.cc:500] Master 127.0.72.188:33967 was elected leader, sending a full tablet report...
I20261118 14:58:09.215839 742 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_b1a3822c61cb4043b4b9ddeaf29cc405" } new_table_name: "default.Table_537f6e87a8d040a695fda278ff06cbf5" modify_external_catalogs: true
W20261118 14:58:09.227422 1259 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:09.252136 743 catalog_manager.cc:2215] created HMS catalog entry for table [id=e26c2363b15444a5882d6ccf7e290f8b]
W20261118 14:58:09.255937 740 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_59239feaba874d6890fe8e86d57545e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:09.256016 735 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ced554c65cd4465d9ddab052dc32efcf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:09.256035 739 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_38deaf819a5e430090d96b02cbd9cdc3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:09.256086 741 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7886615812a14a0492f39f2694999785 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:09.255937 736 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_023fd7c6f2b3407da4cbd94af99cda3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:09.257203 738 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_41739c72be484e8fb242c925dd0e2c32 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:09.258075 737 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d908b160aac24b678c5aff465735ea9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:09.258692 734 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_6c4a501fa569491080a3b4988350b5b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:09.270030 743 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_ab0fd883cbec4e0b8af7febddffe78c5" } new_table_name: "default.Table_2f1acae65de24ae9a7a19515d10acf9c" modify_external_catalogs: true
I20261118 14:58:09.267412 919 tablet_service.cc:1500] Processing CreateTablet for tablet 8ca11d1460404efe9fff0aa389651cec (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
W20261118 14:58:09.274343 1124 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:09.275117 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca11d1460404efe9fff0aa389651cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.278800 1056 tablet_service.cc:1500] Processing CreateTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.280324 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f86e0097f84c2fbf81f4a3ede47f0c. 1 dirs total, 0 dirs full, 0 dirs failed
W20261118 14:58:09.282749 986 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:09.282727 1057 tablet_service.cc:1500] Processing CreateTablet for tablet 8ca11d1460404efe9fff0aa389651cec (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.283260 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca11d1460404efe9fff0aa389651cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.286460 918 tablet_service.cc:1500] Processing CreateTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.287024 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f86e0097f84c2fbf81f4a3ede47f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.290845 1192 tablet_service.cc:1500] Processing CreateTablet for tablet 8ca11d1460404efe9fff0aa389651cec (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.291527 1191 tablet_service.cc:1500] Processing CreateTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.292445 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca11d1460404efe9fff0aa389651cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.298636 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f86e0097f84c2fbf81f4a3ede47f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.308378 1325 tablet_bootstrap.cc:492] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.328204 1326 tablet_bootstrap.cc:492] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.331696 1326 tablet_bootstrap.cc:654] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.333175 1326 log.cc:826] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:09.337708 1325 tablet_bootstrap.cc:654] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.339365 1327 tablet_bootstrap.cc:492] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.343114 1325 log.cc:826] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:09.343421 1326 tablet_bootstrap.cc:492] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.343650 1327 tablet_bootstrap.cc:654] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.343708 1326 ts_tablet_manager.cc:1449] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20261118 14:58:09.344944 1327 log.cc:826] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:09.350880 1326 raft_consensus.cc:360] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.351281 1326 raft_consensus.cc:386] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.351392 1326 raft_consensus.cc:748] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.352187 1326 consensus_queue.cc:261] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.352842 1325 tablet_bootstrap.cc:492] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.353150 1325 ts_tablet_manager.cc:1449] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.045s user 0.007s sys 0.000s
I20261118 14:58:09.354722 740 catalog_manager.cc:2215] created HMS catalog entry for table [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:09.361491 740 master_service.cc:965] ReplaceTablet: received request to replace tablet 8ca11d1460404efe9fff0aa389651cec from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:09.368636 1325 raft_consensus.cc:360] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.369032 1325 raft_consensus.cc:386] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.369129 1325 raft_consensus.cc:748] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.369511 1327 tablet_bootstrap.cc:492] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.369797 1327 ts_tablet_manager.cc:1449] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.004s
I20261118 14:58:09.369808 1325 consensus_queue.cc:261] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.372505 740 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet 8ca11d1460404efe9fff0aa389651cec
I20261118 14:58:09.373632 1191 tablet_service.cc:1547] Processing DeleteTablet for tablet 8ca11d1460404efe9fff0aa389651cec with delete_type TABLET_DATA_DELETED (replaced by tablet 07d0bc6b6bb6484892091f81ecdc505a) from {username='slave'} at 127.0.0.1:60862
I20261118 14:58:09.377350 1327 raft_consensus.cc:360] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.377722 1327 raft_consensus.cc:386] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.377833 1327 raft_consensus.cc:748] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.380304 740 catalog_manager.cc:6936] ReplaceTablet: tablet 8ca11d1460404efe9fff0aa389651cec deleted and replaced by tablet 07d0bc6b6bb6484892091f81ecdc505a
I20261118 14:58:09.382483 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 8ca11d1460404efe9fff0aa389651cec with delete_type TABLET_DATA_DELETED (replaced by tablet 07d0bc6b6bb6484892091f81ecdc505a) from {username='slave'} at 127.0.0.1:49472
I20261118 14:58:09.385407 918 tablet_service.cc:1547] Processing DeleteTablet for tablet 8ca11d1460404efe9fff0aa389651cec with delete_type TABLET_DATA_DELETED (replaced by tablet 07d0bc6b6bb6484892091f81ecdc505a) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:09.386850 819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:09.387116 819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
W20261118 14:58:09.389426 730 catalog_manager.cc:5045] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): delete failed for tablet 8ca11d1460404efe9fff0aa389651cec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ca11d1460404efe9fff0aa389651cec already in progress: creating tablet
I20261118 14:58:09.390641 1327 consensus_queue.cc:261] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
W20261118 14:58:09.394539 727 catalog_manager.cc:5045] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet 8ca11d1460404efe9fff0aa389651cec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ca11d1460404efe9fff0aa389651cec already in progress: creating tablet
I20261118 14:58:09.395558 1327 ts_tablet_manager.cc:1480] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.026s user 0.006s sys 0.009s
I20261118 14:58:09.398123 1327 tablet_bootstrap.cc:492] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.399145 1192 tablet_service.cc:1500] Processing CreateTablet for tablet f66c4e0f06f043108a6451666a80a5f9 (DEFAULT_TABLE table=default.table_59239feaba874d6890fe8e86d57545e2 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.399364 1057 tablet_service.cc:1500] Processing CreateTablet for tablet f66c4e0f06f043108a6451666a80a5f9 (DEFAULT_TABLE table=default.table_59239feaba874d6890fe8e86d57545e2 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.399544 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66c4e0f06f043108a6451666a80a5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.399755 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66c4e0f06f043108a6451666a80a5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.400113 1327 tablet_bootstrap.cc:654] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.403563 918 tablet_service.cc:1500] Processing CreateTablet for tablet f66c4e0f06f043108a6451666a80a5f9 (DEFAULT_TABLE table=default.table_59239feaba874d6890fe8e86d57545e2 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.404042 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66c4e0f06f043108a6451666a80a5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.409531 1327 tablet_bootstrap.cc:492] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.409677 1327 ts_tablet_manager.cc:1449] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20261118 14:58:09.410760 1327 raft_consensus.cc:360] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.410943 1327 raft_consensus.cc:386] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.411018 1327 raft_consensus.cc:748] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.411233 1327 consensus_queue.cc:261] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.411705 1327 ts_tablet_manager.cc:1480] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261118 14:58:09.422230 1190 tablet_service.cc:1500] Processing CreateTablet for tablet 07d0bc6b6bb6484892091f81ecdc505a (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.422751 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d0bc6b6bb6484892091f81ecdc505a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.427938 919 tablet_service.cc:1500] Processing CreateTablet for tablet 07d0bc6b6bb6484892091f81ecdc505a (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.428360 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d0bc6b6bb6484892091f81ecdc505a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.437206 1327 tablet_bootstrap.cc:492] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
W20261118 14:58:09.438980 1260 tablet.cc:2441] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 14:58:09.439218 1327 tablet_bootstrap.cc:654] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.440387 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 8ca11d1460404efe9fff0aa389651cec with delete_type TABLET_DATA_DELETED (replaced by tablet 07d0bc6b6bb6484892091f81ecdc505a) from {username='slave'} at 127.0.0.1:49472
W20261118 14:58:09.441113 730 catalog_manager.cc:5045] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): delete failed for tablet 8ca11d1460404efe9fff0aa389651cec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8ca11d1460404efe9fff0aa389651cec already in progress: creating tablet
I20261118 14:58:09.445015 1057 tablet_service.cc:1500] Processing CreateTablet for tablet 07d0bc6b6bb6484892091f81ecdc505a (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.445539 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d0bc6b6bb6484892091f81ecdc505a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.456141 1327 tablet_bootstrap.cc:492] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.456578 1327 ts_tablet_manager.cc:1449] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.001s
I20261118 14:58:09.457337 740 master_service.cc:965] ReplaceTablet: received request to replace tablet 95f86e0097f84c2fbf81f4a3ede47f0c from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:09.457859 1327 raft_consensus.cc:360] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.458091 1327 raft_consensus.cc:386] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.458171 1327 raft_consensus.cc:748] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.458699 1327 consensus_queue.cc:261] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.459242 1327 ts_tablet_manager.cc:1480] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20261118 14:58:09.460094 1326 ts_tablet_manager.cc:1480] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.116s user 0.014s sys 0.000s
I20261118 14:58:09.463656 1327 tablet_bootstrap.cc:492] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.465166 1326 tablet_bootstrap.cc:492] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.465456 1327 tablet_bootstrap.cc:654] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.470109 1327 tablet_bootstrap.cc:492] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.470417 1327 ts_tablet_manager.cc:1449] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20261118 14:58:09.471236 1327 raft_consensus.cc:360] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.471429 1327 raft_consensus.cc:386] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.471513 1327 raft_consensus.cc:748] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.471736 1327 consensus_queue.cc:261] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.472244 1327 ts_tablet_manager.cc:1480] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20261118 14:58:09.474501 740 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet 95f86e0097f84c2fbf81f4a3ede47f0c
W20261118 14:58:09.474534 1125 tablet.cc:2441] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 14:58:09.476505 1326 tablet_bootstrap.cc:654] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.478068 1333 tablet_replica.cc:333] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:09.478615 1333 raft_consensus.cc:2261] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:09.479570 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c with delete_type TABLET_DATA_DELETED (replaced by tablet dc2c699d9f4746c3905667c67d20e987) from {username='slave'} at 127.0.0.1:49472
I20261118 14:58:09.479941 918 tablet_service.cc:1547] Processing DeleteTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c with delete_type TABLET_DATA_DELETED (replaced by tablet dc2c699d9f4746c3905667c67d20e987) from {username='slave'} at 127.0.0.1:54848
W20261118 14:58:09.480707 730 catalog_manager.cc:5045] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): delete failed for tablet 95f86e0097f84c2fbf81f4a3ede47f0c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95f86e0097f84c2fbf81f4a3ede47f0c already in progress: creating tablet
W20261118 14:58:09.481232 727 catalog_manager.cc:5045] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet 95f86e0097f84c2fbf81f4a3ede47f0c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 95f86e0097f84c2fbf81f4a3ede47f0c already in progress: creating tablet
I20261118 14:58:09.482218 740 catalog_manager.cc:6936] ReplaceTablet: tablet 95f86e0097f84c2fbf81f4a3ede47f0c deleted and replaced by tablet dc2c699d9f4746c3905667c67d20e987
I20261118 14:58:09.480746 1191 tablet_service.cc:1547] Processing DeleteTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c with delete_type TABLET_DATA_DELETED (replaced by tablet dc2c699d9f4746c3905667c67d20e987) from {username='slave'} at 127.0.0.1:60862
I20261118 14:58:09.487706 1056 tablet_service.cc:1500] Processing CreateTablet for tablet dc2c699d9f4746c3905667c67d20e987 (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.488083 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc2c699d9f4746c3905667c67d20e987. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.491236 1191 tablet_service.cc:1500] Processing CreateTablet for tablet dc2c699d9f4746c3905667c67d20e987 (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.491708 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc2c699d9f4746c3905667c67d20e987. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.495570 739 catalog_manager.cc:2215] created HMS catalog entry for table [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:09.496284 918 tablet_service.cc:1500] Processing CreateTablet for tablet dc2c699d9f4746c3905667c67d20e987 (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.496728 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc2c699d9f4746c3905667c67d20e987. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.497575 1327 tablet_bootstrap.cc:492] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.499300 1327 tablet_bootstrap.cc:654] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.499902 1333 raft_consensus.cc:2290] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:09.505693 740 master_service.cc:965] ReplaceTablet: received request to replace tablet 95f86e0097f84c2fbf81f4a3ede47f0c from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:09.507361 1326 tablet_bootstrap.cc:492] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.509321 1326 ts_tablet_manager.cc:1449] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.000s
I20261118 14:58:09.506538 739 master_service.cc:965] ReplaceTablet: received request to replace tablet 8ca11d1460404efe9fff0aa389651cec from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:09.509927 739 master_service.cc:965] ReplaceTablet: received request to replace tablet 95f86e0097f84c2fbf81f4a3ede47f0c from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:09.511557 739 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_59239feaba874d6890fe8e86d57545e2" } new_table_name: "default.Table_a69dc21b9f4948ffad3423d68bff8f34" modify_external_catalogs: true
I20261118 14:58:09.513712 1333 ts_tablet_manager.cc:1962] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:09.515717 919 tablet_service.cc:1500] Processing CreateTablet for tablet 0bbe34e7bb77494a9c9a1e932af6adb8 (DEFAULT_TABLE table=default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.516211 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbe34e7bb77494a9c9a1e932af6adb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.523379 1333 ts_tablet_manager.cc:1975] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:09.528371 1326 raft_consensus.cc:360] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.528592 1326 raft_consensus.cc:386] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.528841 1326 raft_consensus.cc:748] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.529217 1326 consensus_queue.cc:261] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.530992 1326 ts_tablet_manager.cc:1480] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20261118 14:58:09.531313 1326 tablet_bootstrap.cc:492] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.508630 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 8ca11d1460404efe9fff0aa389651cec with delete_type TABLET_DATA_DELETED (replaced by tablet 07d0bc6b6bb6484892091f81ecdc505a) from {username='slave'} at 127.0.0.1:49472
I20261118 14:58:09.525454 1056 tablet_service.cc:1500] Processing CreateTablet for tablet 0bbe34e7bb77494a9c9a1e932af6adb8 (DEFAULT_TABLE table=default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.533354 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbe34e7bb77494a9c9a1e932af6adb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.538662 1334 tablet_replica.cc:333] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:09.539206 1334 raft_consensus.cc:2261] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:09.539651 1334 raft_consensus.cc:2290] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:09.541579 1334 ts_tablet_manager.cc:1962] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:09.546276 1331 raft_consensus.cc:499] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:09.546521 1331 raft_consensus.cc:521] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.548280 1334 ts_tablet_manager.cc:1975] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:09.548440 1334 log.cc:1199] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/8ca11d1460404efe9fff0aa389651cec
I20261118 14:58:09.548661 1333 log.cc:1199] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/8ca11d1460404efe9fff0aa389651cec
I20261118 14:58:09.548661 1331 leader_election.cc:290] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:09.548859 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c with delete_type TABLET_DATA_DELETED (replaced by tablet dc2c699d9f4746c3905667c67d20e987) from {username='slave'} at 127.0.0.1:49472
I20261118 14:58:09.524894 1191 tablet_service.cc:1500] Processing CreateTablet for tablet 0bbe34e7bb77494a9c9a1e932af6adb8 (DEFAULT_TABLE table=default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.549336 1333 ts_tablet_manager.cc:1996] T 8ca11d1460404efe9fff0aa389651cec P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:09.557696 939 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "95f86e0097f84c2fbf81f4a3ede47f0c" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:09.549677 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bbe34e7bb77494a9c9a1e932af6adb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.558306 939 raft_consensus.cc:2490] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:09.559201 1010 leader_election.cc:304] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:09.559578 1331 raft_consensus.cc:2826] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:09.559719 1331 raft_consensus.cc:499] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:09.559796 1331 raft_consensus.cc:3096] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.562568 1331 raft_consensus.cc:521] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.563560 939 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "95f86e0097f84c2fbf81f4a3ede47f0c" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:09.563764 939 raft_consensus.cc:3096] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.566881 939 raft_consensus.cc:2490] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:09.567312 1010 leader_election.cc:304] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:09.568372 740 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:09.568898 1344 raft_consensus.cc:2826] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:09.570772 1344 raft_consensus.cc:705] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:09.571112 1325 ts_tablet_manager.cc:1480] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.218s user 0.057s sys 0.106s
I20261118 14:58:09.571283 1344 consensus_queue.cc:238] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.572654 1331 leader_election.cc:290] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:09.573918 729 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 8ca11d1460404efe9fff0aa389651cec (table default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:09.575659 1326 tablet_bootstrap.cc:654] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.576700 1334 ts_tablet_manager.cc:1996] T 8ca11d1460404efe9fff0aa389651cec P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:09.576990 1333 tablet_replica.cc:333] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:09.577224 1333 raft_consensus.cc:2261] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:09.577445 1333 raft_consensus.cc:2290] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:09.578222 1334 tablet_replica.cc:333] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:09.578393 730 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 8ca11d1460404efe9fff0aa389651cec (table default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:09.578934 1333 ts_tablet_manager.cc:1962] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:09.579303 1325 tablet_bootstrap.cc:492] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.581411 1325 tablet_bootstrap.cc:654] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.582011 1327 tablet_bootstrap.cc:492] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.582188 1327 ts_tablet_manager.cc:1449] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.085s user 0.012s sys 0.022s
I20261118 14:58:09.582969 1327 raft_consensus.cc:360] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.583196 1327 raft_consensus.cc:386] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.583273 1327 raft_consensus.cc:748] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.584220 1327 consensus_queue.cc:261] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.590847 1326 tablet_bootstrap.cc:492] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.591023 1326 ts_tablet_manager.cc:1449] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.060s user 0.002s sys 0.004s
I20261118 14:58:09.591773 1326 raft_consensus.cc:360] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.591966 1326 raft_consensus.cc:386] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.592079 1326 raft_consensus.cc:748] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.592304 1327 ts_tablet_manager.cc:1480] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20261118 14:58:09.592375 1326 consensus_queue.cc:261] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.592476 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "95f86e0097f84c2fbf81f4a3ede47f0c" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:09.592914 1326 ts_tablet_manager.cc:1480] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 14:58:09.593345 1326 tablet_bootstrap.cc:492] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.593493 1212 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "95f86e0097f84c2fbf81f4a3ede47f0c" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:09.593667 1327 tablet_bootstrap.cc:492] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.595489 1327 tablet_bootstrap.cc:654] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
W20261118 14:58:09.596042 1012 leader_election.cc:343] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Illegal state: Tablet not RUNNING: STOPPED
W20261118 14:58:09.596753 1012 leader_election.cc:343] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Illegal state: Tablet not RUNNING: STOPPED
I20261118 14:58:09.597056 1334 raft_consensus.cc:2261] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:09.597838 1334 pending_rounds.cc:70] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Trying to abort 1 pending ops.
I20261118 14:58:09.598016 1334 pending_rounds.cc:77] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7352376891717300224 op_type: NO_OP noop_request { }
I20261118 14:58:09.598212 1334 raft_consensus.cc:2917] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20261118 14:58:09.598387 1334 raft_consensus.cc:2290] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:09.598872 1333 ts_tablet_manager.cc:1975] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:09.598999 1333 log.cc:1199] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/95f86e0097f84c2fbf81f4a3ede47f0c
I20261118 14:58:09.599157 1326 tablet_bootstrap.cc:654] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.599309 1333 ts_tablet_manager.cc:1996] T 95f86e0097f84c2fbf81f4a3ede47f0c P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:09.601142 1334 ts_tablet_manager.cc:1962] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:09.602636 729 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 95f86e0097f84c2fbf81f4a3ede47f0c (table default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:09.603924 1325 tablet_bootstrap.cc:492] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.604091 1325 ts_tablet_manager.cc:1449] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.008s
I20261118 14:58:09.605043 1325 raft_consensus.cc:360] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.605247 1325 raft_consensus.cc:386] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.605343 1325 raft_consensus.cc:748] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.605647 1325 consensus_queue.cc:261] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.606172 1325 ts_tablet_manager.cc:1480] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20261118 14:58:09.606947 1325 tablet_bootstrap.cc:492] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.608704 1325 tablet_bootstrap.cc:654] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.613216 1326 tablet_bootstrap.cc:492] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.613395 1334 ts_tablet_manager.cc:1975] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:09.613517 1334 log.cc:1199] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/95f86e0097f84c2fbf81f4a3ede47f0c
I20261118 14:58:09.613833 1334 ts_tablet_manager.cc:1996] T 95f86e0097f84c2fbf81f4a3ede47f0c P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:09.613425 1326 ts_tablet_manager.cc:1449] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.006s
I20261118 14:58:09.614486 1325 tablet_bootstrap.cc:492] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.614642 1325 ts_tablet_manager.cc:1449] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20261118 14:58:09.614909 1326 raft_consensus.cc:360] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.615075 1326 raft_consensus.cc:386] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.615170 1326 raft_consensus.cc:748] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.615370 1325 raft_consensus.cc:360] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.615386 1326 consensus_queue.cc:261] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.615885 1326 ts_tablet_manager.cc:1480] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20261118 14:58:09.616510 1326 tablet_bootstrap.cc:492] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.615558 1325 raft_consensus.cc:386] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.618600 1325 raft_consensus.cc:748] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.618997 1325 consensus_queue.cc:261] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.619069 730 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 95f86e0097f84c2fbf81f4a3ede47f0c (table default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:09.619328 741 catalog_manager.cc:2215] created HMS catalog entry for table [id=fae4a508ca0b4035a4843cdcfa56f4e7]
I20261118 14:58:09.620561 1325 ts_tablet_manager.cc:1480] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20261118 14:58:09.620901 1325 tablet_bootstrap.cc:492] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.622768 1325 tablet_bootstrap.cc:654] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.624946 1326 tablet_bootstrap.cc:654] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.628553 1327 tablet_bootstrap.cc:492] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.628728 1327 ts_tablet_manager.cc:1449] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.006s
I20261118 14:58:09.629547 1327 raft_consensus.cc:360] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.629747 1327 raft_consensus.cc:386] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.629835 1327 raft_consensus.cc:748] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.630093 1327 consensus_queue.cc:261] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.633870 1327 ts_tablet_manager.cc:1480] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20261118 14:58:09.634032 918 tablet_service.cc:1547] Processing DeleteTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c with delete_type TABLET_DATA_DELETED (replaced by tablet dc2c699d9f4746c3905667c67d20e987) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:09.634994 1335 tablet_replica.cc:333] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:09.635478 1335 raft_consensus.cc:2261] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:09.635941 1335 raft_consensus.cc:2290] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:09.642086 1335 ts_tablet_manager.cc:1962] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:09.657006 1191 tablet_service.cc:1500] Processing CreateTablet for tablet a698428dd3b14661be8e17bac5fd56f1 (DEFAULT_TABLE table=default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.657557 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a698428dd3b14661be8e17bac5fd56f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.663244 1056 tablet_service.cc:1500] Processing CreateTablet for tablet a698428dd3b14661be8e17bac5fd56f1 (DEFAULT_TABLE table=default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.663739 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a698428dd3b14661be8e17bac5fd56f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.663645 918 tablet_service.cc:1500] Processing CreateTablet for tablet a698428dd3b14661be8e17bac5fd56f1 (DEFAULT_TABLE table=default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.664058 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a698428dd3b14661be8e17bac5fd56f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.670140 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 95f86e0097f84c2fbf81f4a3ede47f0c with delete_type TABLET_DATA_DELETED (replaced by tablet dc2c699d9f4746c3905667c67d20e987) from {username='slave'} at 127.0.0.1:49472
W20261118 14:58:09.670930 730 catalog_manager.cc:5032] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): delete failed for tablet 95f86e0097f84c2fbf81f4a3ede47f0c because the tablet was not found. No further retry: Not found: Tablet not found: 95f86e0097f84c2fbf81f4a3ede47f0c
I20261118 14:58:09.673450 919 tablet_service.cc:1547] Processing DeleteTablet for tablet 8ca11d1460404efe9fff0aa389651cec with delete_type TABLET_DATA_DELETED (replaced by tablet 07d0bc6b6bb6484892091f81ecdc505a) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:09.684006 741 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:09.688355 1327 tablet_bootstrap.cc:492] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.690189 1327 tablet_bootstrap.cc:654] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.695225 1335 ts_tablet_manager.cc:1975] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:09.695473 1335 log.cc:1199] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/95f86e0097f84c2fbf81f4a3ede47f0c
I20261118 14:58:09.696028 1335 ts_tablet_manager.cc:1996] T 95f86e0097f84c2fbf81f4a3ede47f0c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:09.716616 727 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 95f86e0097f84c2fbf81f4a3ede47f0c (table default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:09.717058 1335 tablet_replica.cc:333] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:09.717350 1335 raft_consensus.cc:2261] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:09.717684 1335 raft_consensus.cc:2290] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Raft consensus is shut down!
W20261118 14:58:09.718134 987 tablet.cc:2441] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 14:58:09.722555 736 catalog_manager.cc:2215] created HMS catalog entry for table [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:09.729491 736 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_38deaf819a5e430090d96b02cbd9cdc3" } new_table_name: "default.Table_3f41e3f42a794945838124ce6712ceb3" modify_external_catalogs: true
I20261118 14:58:09.741888 1191 tablet_service.cc:1500] Processing CreateTablet for tablet e47d7d10a5c741ecb30ec701c76b27a3 (DEFAULT_TABLE table=default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.742487 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47d7d10a5c741ecb30ec701c76b27a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.743201 1056 tablet_service.cc:1500] Processing CreateTablet for tablet e47d7d10a5c741ecb30ec701c76b27a3 (DEFAULT_TABLE table=default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.743774 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47d7d10a5c741ecb30ec701c76b27a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.746711 918 tablet_service.cc:1500] Processing CreateTablet for tablet e47d7d10a5c741ecb30ec701c76b27a3 (DEFAULT_TABLE table=default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.747151 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47d7d10a5c741ecb30ec701c76b27a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.748344 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:46707
--webserver_interface=127.0.72.189
--webserver_port=42561
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 14:58:09.761559 1335 ts_tablet_manager.cc:1962] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:09.761634 1327 tablet_bootstrap.cc:492] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.761786 1327 ts_tablet_manager.cc:1449] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.074s user 0.007s sys 0.006s
I20261118 14:58:09.762672 1327 raft_consensus.cc:360] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.762871 1327 raft_consensus.cc:386] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.762944 1327 raft_consensus.cc:748] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.763178 1327 consensus_queue.cc:261] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.763671 1327 ts_tablet_manager.cc:1480] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:09.763989 1327 tablet_bootstrap.cc:492] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.765605 1327 tablet_bootstrap.cc:654] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.767277 1326 tablet_bootstrap.cc:492] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.767462 1326 ts_tablet_manager.cc:1449] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.151s user 0.015s sys 0.043s
I20261118 14:58:09.768208 1326 raft_consensus.cc:360] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.768414 1326 raft_consensus.cc:386] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.768493 1326 raft_consensus.cc:748] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.768806 1326 consensus_queue.cc:261] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.769280 1326 ts_tablet_manager.cc:1480] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20261118 14:58:09.769567 1326 tablet_bootstrap.cc:492] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.771888 1326 tablet_bootstrap.cc:654] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.779465 1325 tablet_bootstrap.cc:492] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.779609 1325 ts_tablet_manager.cc:1449] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.159s user 0.075s sys 0.068s
I20261118 14:58:09.780321 1325 raft_consensus.cc:360] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.780496 1325 raft_consensus.cc:386] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.780577 1325 raft_consensus.cc:748] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.780848 1325 consensus_queue.cc:261] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.784019 1327 tablet_bootstrap.cc:492] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.784176 1327 ts_tablet_manager.cc:1449] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.007s
I20261118 14:58:09.784617 1325 ts_tablet_manager.cc:1480] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20261118 14:58:09.785079 1327 raft_consensus.cc:360] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.785287 1327 raft_consensus.cc:386] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.785377 1327 raft_consensus.cc:748] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.785634 1327 consensus_queue.cc:261] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.786372 1326 tablet_bootstrap.cc:492] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.786530 1326 ts_tablet_manager.cc:1449] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20261118 14:58:09.786944 1325 tablet_bootstrap.cc:492] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.788287 1326 raft_consensus.cc:360] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.788476 1326 raft_consensus.cc:386] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.788699 1325 tablet_bootstrap.cc:654] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.789680 1327 ts_tablet_manager.cc:1480] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20261118 14:58:09.788573 1326 raft_consensus.cc:748] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.790829 1326 consensus_queue.cc:261] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.791337 1326 ts_tablet_manager.cc:1480] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20261118 14:58:09.791714 1326 tablet_bootstrap.cc:492] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.792722 1335 ts_tablet_manager.cc:1975] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:09.792907 1335 log.cc:1199] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/8ca11d1460404efe9fff0aa389651cec
I20261118 14:58:09.793339 1335 ts_tablet_manager.cc:1996] T 8ca11d1460404efe9fff0aa389651cec P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:09.793375 1326 tablet_bootstrap.cc:654] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.801788 1325 tablet_bootstrap.cc:492] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.801940 1325 ts_tablet_manager.cc:1449] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.009s
I20261118 14:58:09.802068 727 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 8ca11d1460404efe9fff0aa389651cec (table default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:09.802618 1325 raft_consensus.cc:360] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.802812 1325 raft_consensus.cc:386] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.802884 1325 raft_consensus.cc:748] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.803236 1325 consensus_queue.cc:261] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.805410 1325 ts_tablet_manager.cc:1480] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:09.807337 1325 tablet_bootstrap.cc:492] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.808923 1326 tablet_bootstrap.cc:492] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.808944 1325 tablet_bootstrap.cc:654] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.809080 1326 ts_tablet_manager.cc:1449] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20261118 14:58:09.809820 1326 raft_consensus.cc:360] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.810021 1326 raft_consensus.cc:386] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.810099 1326 raft_consensus.cc:748] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.810566 1326 consensus_queue.cc:261] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.812590 1326 ts_tablet_manager.cc:1480] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:09.812892 1326 tablet_bootstrap.cc:492] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.815160 1326 tablet_bootstrap.cc:654] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.822853 1325 tablet_bootstrap.cc:492] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.823007 1325 ts_tablet_manager.cc:1449] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.008s
I20261118 14:58:09.823745 1325 raft_consensus.cc:360] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.823945 1325 raft_consensus.cc:386] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.824033 1325 raft_consensus.cc:748] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.824308 1325 consensus_queue.cc:261] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.826717 1325 ts_tablet_manager.cc:1480] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20261118 14:58:09.828166 735 catalog_manager.cc:2215] created HMS catalog entry for table [id=e4b541d82e0f4d0991446b865e206f8f]
I20261118 14:58:09.828768 1325 tablet_bootstrap.cc:492] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.830665 1325 tablet_bootstrap.cc:654] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.835973 735 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:39006 (request call id 9) took 1084 ms. Trace:
I20261118 14:58:09.836122 735 rpcz_store.cc:276] 1118 14:58:08.751614 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:08.751823 (+ 209us) service_pool.cc:224] Handling call
1118 14:58:09.257075 (+505252us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:09.257531 (+ 456us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:09.828119 (+570588us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:09.835710 (+ 7591us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 14:58:09.835828 (+ 118us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 14:58:09.835937 (+ 109us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:09.828869 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 14:58:09.828995 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
1118 14:58:09.829002 (+ 7us) write_op.cc:435] Acquired schema lock
1118 14:58:09.829004 (+ 2us) tablet.cc:670] Decoding operations
1118 14:58:09.829108 (+ 104us) write_op.cc:620] Acquiring the partition lock for write op
1118 14:58:09.829123 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1118 14:58:09.829128 (+ 5us) tablet.cc:693] Acquiring locks for 2 operations
1118 14:58:09.829208 (+ 80us) tablet.cc:709] Row locks acquired
1118 14:58:09.829210 (+ 2us) write_op.cc:260] PREPARE: finished
1118 14:58:09.829247 (+ 37us) write_op.cc:270] Start()
1118 14:58:09.829281 (+ 34us) write_op.cc:276] Timestamp: P: 1795013889829222 usec, L: 0
1118 14:58:09.829285 (+ 4us) op_driver.cc:350] REPLICATION: starting
1118 14:58:09.829482 (+ 197us) log.cc:844] Serialized 432 byte log entry
1118 14:58:09.834809 (+ 5327us) op_driver.cc:466] REPLICATION: finished
1118 14:58:09.834943 (+ 134us) write_op.cc:301] APPLY: starting
1118 14:58:09.834960 (+ 17us) tablet.cc:1402] starting BulkCheckPresence
1118 14:58:09.835006 (+ 46us) tablet.cc:1405] finished BulkCheckPresence
1118 14:58:09.835038 (+ 32us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 14:58:09.835139 (+ 101us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 14:58:09.835175 (+ 36us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 14:58:09.835183 (+ 8us) write_op.cc:312] APPLY: finished
1118 14:58:09.835257 (+ 74us) log.cc:844] Serialized 28 byte log entry
1118 14:58:09.835318 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:09.835337 (+ 19us) write_op.cc:454] Released schema lock
1118 14:58:09.835343 (+ 6us) write_op.cc:341] FINISH: Updating metrics
1118 14:58:09.835408 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:09.835412 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":464601,"Hive Metastore.run_cpu_time_us":284,"Hive Metastore.run_wall_time_us":105570,"spinlock_wait_cycles":262656,"child_traces":[["op",{"apply.queue_time_us":85,"apply.run_cpu_time_us":459,"apply.run_wall_time_us":457,"num_ops":2,"prepare.queue_time_us":84,"prepare.run_cpu_time_us":717,"prepare.run_wall_time_us":716,"raft.queue_time_us":1098,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":284,"replication_time_us":5491}]]}
I20261118 14:58:09.838668 1325 tablet_bootstrap.cc:492] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.838827 1325 ts_tablet_manager.cc:1449] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20261118 14:58:09.840607 1325 raft_consensus.cc:360] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.840806 1325 raft_consensus.cc:386] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.840873 1325 raft_consensus.cc:748] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.841162 1325 consensus_queue.cc:261] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.843607 1325 ts_tablet_manager.cc:1480] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20261118 14:58:09.845804 1325 tablet_bootstrap.cc:492] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.847589 1325 tablet_bootstrap.cc:654] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.848500 1056 tablet_service.cc:1500] Processing CreateTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 (DEFAULT_TABLE table=default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.848933 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a07aac51e8a48dba2c61e9e477fad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.849270 1191 tablet_service.cc:1500] Processing CreateTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 (DEFAULT_TABLE table=default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.849661 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a07aac51e8a48dba2c61e9e477fad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.852384 735 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_9021a49aff8546eca95c57bde20822b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:09.852630 918 tablet_service.cc:1500] Processing CreateTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 (DEFAULT_TABLE table=default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.852993 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a07aac51e8a48dba2c61e9e477fad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.862759 1337 raft_consensus.cc:499] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:09.863029 1337 raft_consensus.cc:521] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.864676 1327 tablet_bootstrap.cc:492] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.865026 1337 leader_election.cc:290] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:09.866501 1327 tablet_bootstrap.cc:654] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.871369 1326 tablet_bootstrap.cc:492] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.872200 1326 ts_tablet_manager.cc:1449] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.029s
I20261118 14:58:09.873296 1326 raft_consensus.cc:360] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.874290 939 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "07d0bc6b6bb6484892091f81ecdc505a" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:09.874541 939 raft_consensus.cc:2490] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:09.874574 1326 raft_consensus.cc:386] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.874938 1326 raft_consensus.cc:748] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.875088 1145 leader_election.cc:304] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:09.875522 1337 raft_consensus.cc:2826] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:09.875662 1337 raft_consensus.cc:499] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:09.875762 1337 raft_consensus.cc:3096] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.875479 1326 consensus_queue.cc:261] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.879462 1077 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "07d0bc6b6bb6484892091f81ecdc505a" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:09.879840 1077 raft_consensus.cc:2490] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:09.879047 1326 ts_tablet_manager.cc:1480] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20261118 14:58:09.883147 1326 tablet_bootstrap.cc:492] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.884377 1337 raft_consensus.cc:521] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.885469 939 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "07d0bc6b6bb6484892091f81ecdc505a" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:09.885699 939 raft_consensus.cc:3096] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.886588 1326 tablet_bootstrap.cc:654] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.888736 939 raft_consensus.cc:2490] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:09.889181 1145 leader_election.cc:304] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:09.890475 1327 tablet_bootstrap.cc:492] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:09.890713 1327 ts_tablet_manager.cc:1449] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.008s
I20261118 14:58:09.891474 1337 leader_election.cc:290] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:09.891698 1337 raft_consensus.cc:2826] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:09.891873 1337 raft_consensus.cc:705] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:09.891891 1077 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "07d0bc6b6bb6484892091f81ecdc505a" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:09.892046 1077 raft_consensus.cc:3096] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.892331 1327 raft_consensus.cc:360] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.892742 1327 raft_consensus.cc:386] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.892859 1327 raft_consensus.cc:748] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.893131 1327 consensus_queue.cc:261] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.892341 1337 consensus_queue.cc:238] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:09.894858 1077 raft_consensus.cc:2490] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:09.895980 1326 tablet_bootstrap.cc:492] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:09.896148 1326 ts_tablet_manager.cc:1449] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.002s
I20261118 14:58:09.896817 1326 raft_consensus.cc:360] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.897006 1326 raft_consensus.cc:386] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.897076 1326 raft_consensus.cc:748] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.897349 1326 consensus_queue.cc:261] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.897814 1326 ts_tablet_manager.cc:1480] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 14:58:09.913057 1327 ts_tablet_manager.cc:1480] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.022s user 0.005s sys 0.002s
I20261118 14:58:09.925124 738 catalog_manager.cc:2215] created HMS catalog entry for table [id=1b827df58389480588f081062466b985]
I20261118 14:58:09.927906 1368 consensus_queue.cc:1049] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20261118 14:58:09.933169 1368 consensus_queue.cc:1049] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:09.937211 738 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:39006 (request call id 6) took 1189 ms. Trace:
I20261118 14:58:09.937352 738 rpcz_store.cc:276] 1118 14:58:08.747241 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:08.747287 (+ 46us) service_pool.cc:224] Handling call
1118 14:58:09.257325 (+510038us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:09.257736 (+ 411us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:09.925073 (+667337us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:09.936907 (+ 11834us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 14:58:09.937051 (+ 144us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 14:58:09.937171 (+ 120us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:09.925922 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 14:58:09.926056 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
1118 14:58:09.926063 (+ 7us) write_op.cc:435] Acquired schema lock
1118 14:58:09.926066 (+ 3us) tablet.cc:670] Decoding operations
1118 14:58:09.926162 (+ 96us) write_op.cc:620] Acquiring the partition lock for write op
1118 14:58:09.926176 (+ 14us) write_op.cc:641] Partition lock acquired for write op
1118 14:58:09.926181 (+ 5us) tablet.cc:693] Acquiring locks for 2 operations
1118 14:58:09.926261 (+ 80us) tablet.cc:709] Row locks acquired
1118 14:58:09.926263 (+ 2us) write_op.cc:260] PREPARE: finished
1118 14:58:09.926300 (+ 37us) write_op.cc:270] Start()
1118 14:58:09.926334 (+ 34us) write_op.cc:276] Timestamp: P: 1795013889926265 usec, L: 0
1118 14:58:09.926338 (+ 4us) op_driver.cc:350] REPLICATION: starting
1118 14:58:09.926540 (+ 202us) log.cc:844] Serialized 432 byte log entry
1118 14:58:09.932068 (+ 5528us) op_driver.cc:466] REPLICATION: finished
1118 14:58:09.932974 (+ 906us) write_op.cc:301] APPLY: starting
1118 14:58:09.932993 (+ 19us) tablet.cc:1402] starting BulkCheckPresence
1118 14:58:09.933062 (+ 69us) tablet.cc:1405] finished BulkCheckPresence
1118 14:58:09.933102 (+ 40us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 14:58:09.933225 (+ 123us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 14:58:09.933262 (+ 37us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 14:58:09.933271 (+ 9us) write_op.cc:312] APPLY: finished
1118 14:58:09.933361 (+ 90us) log.cc:844] Serialized 28 byte log entry
1118 14:58:09.933426 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:09.933450 (+ 24us) write_op.cc:454] Released schema lock
1118 14:58:09.933457 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1118 14:58:09.933527 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:09.933531 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":570025,"Hive Metastore.run_cpu_time_us":287,"Hive Metastore.run_wall_time_us":96943,"child_traces":[["op",{"apply.queue_time_us":830,"apply.run_cpu_time_us":552,"apply.run_wall_time_us":552,"num_ops":2,"prepare.queue_time_us":92,"prepare.run_cpu_time_us":726,"prepare.run_wall_time_us":727,"raft.queue_time_us":963,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":274,"replication_time_us":5698}]]}
I20261118 14:58:09.943012 1368 raft_consensus.cc:499] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:09.943189 1368 raft_consensus.cc:521] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.944460 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f66c4e0f06f043108a6451666a80a5f9" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:09.944808 938 raft_consensus.cc:2490] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:09.944875 1368 leader_election.cc:290] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:09.945688 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f66c4e0f06f043108a6451666a80a5f9" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:09.945892 1076 raft_consensus.cc:2490] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:09.946313 1148 leader_election.cc:304] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:09.946568 1325 tablet_bootstrap.cc:492] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.946638 1368 raft_consensus.cc:2826] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:09.946718 1325 ts_tablet_manager.cc:1449] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.101s user 0.028s sys 0.038s
I20261118 14:58:09.946754 1368 raft_consensus.cc:499] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:09.946843 1368 raft_consensus.cc:3096] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.947531 1325 raft_consensus.cc:360] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.947717 1325 raft_consensus.cc:386] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.947810 1325 raft_consensus.cc:748] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.948076 1325 consensus_queue.cc:261] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.948607 1325 ts_tablet_manager.cc:1480] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:09.948915 1325 tablet_bootstrap.cc:492] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.950726 1325 tablet_bootstrap.cc:654] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.951318 918 tablet_service.cc:1500] Processing CreateTablet for tablet 91a4a7f508b44f809c8071413df8d648 (DEFAULT_TABLE table=default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.951762 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a4a7f508b44f809c8071413df8d648. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.951886 1368 raft_consensus.cc:521] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.952924 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f66c4e0f06f043108a6451666a80a5f9" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:09.953125 938 raft_consensus.cc:3096] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.954911 1191 tablet_service.cc:1500] Processing CreateTablet for tablet 91a4a7f508b44f809c8071413df8d648 (DEFAULT_TABLE table=default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.955116 1056 tablet_service.cc:1500] Processing CreateTablet for tablet 91a4a7f508b44f809c8071413df8d648 (DEFAULT_TABLE table=default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:09.955310 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a4a7f508b44f809c8071413df8d648. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.955508 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91a4a7f508b44f809c8071413df8d648. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:09.956066 938 raft_consensus.cc:2490] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:09.956498 1145 leader_election.cc:304] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:09.956846 1337 raft_consensus.cc:2826] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:09.956992 1337 raft_consensus.cc:705] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:09.957281 1337 consensus_queue.cc:238] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.961333 1368 leader_election.cc:290] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:09.961719 1327 tablet_bootstrap.cc:492] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:09.963572 1327 tablet_bootstrap.cc:654] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.965509 1325 tablet_bootstrap.cc:492] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:09.965677 1325 ts_tablet_manager.cc:1449] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.006s
I20261118 14:58:09.965648 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f66c4e0f06f043108a6451666a80a5f9" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:09.965831 1076 raft_consensus.cc:3096] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.966446 1325 raft_consensus.cc:360] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.966638 1325 raft_consensus.cc:386] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:09.966699 1325 raft_consensus.cc:748] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:09.966987 1325 consensus_queue.cc:261] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.968648 1325 ts_tablet_manager.cc:1480] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:09.968657 1076 raft_consensus.cc:2490] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:09.970997 738 master_service.cc:965] ReplaceTablet: received request to replace tablet dc2c699d9f4746c3905667c67d20e987 from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:09.977685 1326 tablet_bootstrap.cc:492] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:09.979449 1326 tablet_bootstrap.cc:654] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.980340 1325 tablet_bootstrap.cc:492] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:09.981950 1325 tablet_bootstrap.cc:654] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:09.987366 738 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet dc2c699d9f4746c3905667c67d20e987
I20261118 14:58:09.988220 738 catalog_manager.cc:6936] ReplaceTablet: tablet dc2c699d9f4746c3905667c67d20e987 deleted and replaced by tablet f4bdcca79fcb4fedaea1fdbe2dd31112
I20261118 14:58:09.988473 1344 raft_consensus.cc:499] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:09.988660 1344 raft_consensus.cc:521] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.989387 1344 leader_election.cc:290] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:09.990198 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8a07aac51e8a48dba2c61e9e477fad11" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:09.990480 938 raft_consensus.cc:2490] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:09.990152 738 catalog_manager.cc:5840] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "c01f1b88968f4905ba5a401a81ade9f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } }
I20261118 14:58:09.990979 1010 leader_election.cc:304] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:09.991314 1344 raft_consensus.cc:2826] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:09.991405 1344 raft_consensus.cc:499] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:09.991504 1344 raft_consensus.cc:3096] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.992033 1212 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8a07aac51e8a48dba2c61e9e477fad11" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:09.992394 1212 raft_consensus.cc:2490] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:09.992751 1191 tablet_service.cc:1547] Processing DeleteTablet for tablet dc2c699d9f4746c3905667c67d20e987 with delete_type TABLET_DATA_DELETED (replaced by tablet f4bdcca79fcb4fedaea1fdbe2dd31112) from {username='slave'} at 127.0.0.1:60862
I20261118 14:58:09.993274 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet dc2c699d9f4746c3905667c67d20e987 with delete_type TABLET_DATA_DELETED (replaced by tablet f4bdcca79fcb4fedaea1fdbe2dd31112) from {username='slave'} at 127.0.0.1:49472
I20261118 14:58:09.993353 1333 tablet_replica.cc:333] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:09.993537 1334 tablet_replica.cc:333] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:09.993547 1333 raft_consensus.cc:2261] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:09.993738 1333 raft_consensus.cc:2290] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:09.993754 1334 raft_consensus.cc:2261] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:09.993907 1334 raft_consensus.cc:2290] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:09.994765 1334 ts_tablet_manager.cc:1962] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:09.995456 1344 raft_consensus.cc:521] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:09.996506 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8a07aac51e8a48dba2c61e9e477fad11" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:09.996744 938 raft_consensus.cc:3096] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:09.999482 938 raft_consensus.cc:2490] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:09.999945 1010 leader_election.cc:304] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:10.000289 1345 raft_consensus.cc:2826] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.000449 1345 raft_consensus.cc:705] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:10.000736 1344 leader_election.cc:290] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.000883 918 tablet_service.cc:1547] Processing DeleteTablet for tablet dc2c699d9f4746c3905667c67d20e987 with delete_type TABLET_DATA_DELETED (replaced by tablet f4bdcca79fcb4fedaea1fdbe2dd31112) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:10.001197 1335 tablet_replica.cc:333] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:10.000757 1345 consensus_queue.cc:238] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.001384 1335 raft_consensus.cc:2261] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:10.001557 1335 raft_consensus.cc:2290] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.002406 1335 ts_tablet_manager.cc:1962] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.002547 1212 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8a07aac51e8a48dba2c61e9e477fad11" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:10.002740 1212 raft_consensus.cc:3096] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.005543 1334 ts_tablet_manager.cc:1975] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:10.005669 1334 log.cc:1199] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/dc2c699d9f4746c3905667c67d20e987
I20261118 14:58:10.005726 1212 raft_consensus.cc:2490] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:10.005995 1334 ts_tablet_manager.cc:1996] T dc2c699d9f4746c3905667c67d20e987 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
W20261118 14:58:10.009285 1376 raft_consensus.cc:678] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20261118 14:58:10.009831 1335 ts_tablet_manager.cc:1975] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:10.010020 1335 log.cc:1199] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/dc2c699d9f4746c3905667c67d20e987
I20261118 14:58:10.010350 1333 ts_tablet_manager.cc:1962] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.010680 1332 raft_consensus.cc:499] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.010823 1332 raft_consensus.cc:521] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.011389 1335 ts_tablet_manager.cc:1996] T dc2c699d9f4746c3905667c67d20e987 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:10.012454 727 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet dc2c699d9f4746c3905667c67d20e987 (table default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:10.012791 1332 leader_election.cc:290] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.016656 1333 ts_tablet_manager.cc:1975] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:10.016791 1333 log.cc:1199] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/dc2c699d9f4746c3905667c67d20e987
I20261118 14:58:10.017198 1333 ts_tablet_manager.cc:1996] T dc2c699d9f4746c3905667c67d20e987 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:10.018461 729 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet dc2c699d9f4746c3905667c67d20e987 (table default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:10.020808 1191 tablet_service.cc:1500] Processing CreateTablet for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.021245 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdcca79fcb4fedaea1fdbe2dd31112. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.023351 1212 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0bbe34e7bb77494a9c9a1e932af6adb8" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:10.023571 1212 raft_consensus.cc:2490] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.024120 874 leader_election.cc:304] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.024550 1332 raft_consensus.cc:2826] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.024678 1332 raft_consensus.cc:499] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.024770 1332 raft_consensus.cc:3096] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.027415 1056 tablet_service.cc:1500] Processing CreateTablet for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.027838 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdcca79fcb4fedaea1fdbe2dd31112. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.028805 1332 raft_consensus.cc:521] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.030184 1212 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0bbe34e7bb77494a9c9a1e932af6adb8" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:10.030405 1212 raft_consensus.cc:3096] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.038702 738 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_7886615812a14a0492f39f2694999785" } new_table_name: "default.Table_927d6a72d3524190964efb6e9f1f8d42" modify_external_catalogs: true
I20261118 14:58:10.039434 1212 raft_consensus.cc:2490] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.040932 1332 leader_election.cc:290] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.043066 1326 tablet_bootstrap.cc:492] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:10.043237 1326 ts_tablet_manager.cc:1449] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.066s user 0.007s sys 0.003s
I20261118 14:58:10.044122 1326 raft_consensus.cc:360] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.044299 1326 raft_consensus.cc:386] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.044395 1326 raft_consensus.cc:748] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.044639 1326 consensus_queue.cc:261] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.045475 1326 ts_tablet_manager.cc:1480] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261118 14:58:10.041648 1327 tablet_bootstrap.cc:492] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:10.045790 1327 ts_tablet_manager.cc:1449] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.084s user 0.006s sys 0.006s
I20261118 14:58:10.046571 874 leader_election.cc:304] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.046592 1327 raft_consensus.cc:360] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.046779 1327 raft_consensus.cc:386] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.046864 1327 raft_consensus.cc:748] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.046900 1332 raft_consensus.cc:2826] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.047149 1327 consensus_queue.cc:261] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.047689 1327 ts_tablet_manager.cc:1480] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:10.047981 1327 tablet_bootstrap.cc:492] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:10.049916 1327 tablet_bootstrap.cc:654] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.051015 1326 tablet_bootstrap.cc:492] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:10.052802 1326 tablet_bootstrap.cc:654] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.053274 730 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet dc2c699d9f4746c3905667c67d20e987 (table default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:10.056099 1368 consensus_queue.cc:1049] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.056984 1325 tablet_bootstrap.cc:492] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:10.057194 1369 consensus_queue.cc:1049] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.057407 1325 ts_tablet_manager.cc:1449] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.004s
I20261118 14:58:10.058327 1325 raft_consensus.cc:360] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.067456 1325 raft_consensus.cc:386] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.068240 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0bbe34e7bb77494a9c9a1e932af6adb8" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.068487 1076 raft_consensus.cc:2490] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.068516 1325 raft_consensus.cc:748] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.067420 918 tablet_service.cc:1500] Processing CreateTablet for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (DEFAULT_TABLE table=default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.068929 1325 consensus_queue.cc:261] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.069195 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4bdcca79fcb4fedaea1fdbe2dd31112. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.062939 1332 raft_consensus.cc:705] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:10.075872 1325 ts_tablet_manager.cc:1480] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20261118 14:58:10.076227 1325 tablet_bootstrap.cc:492] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:10.076021 1332 consensus_queue.cc:238] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.080001 1325 tablet_bootstrap.cc:654] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.082108 737 catalog_manager.cc:2215] created HMS catalog entry for table [id=8656240a083440b4b78095548a82a9c7]
I20261118 14:58:10.082255 1326 tablet_bootstrap.cc:492] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:10.082451 1326 ts_tablet_manager.cc:1449] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.005s
I20261118 14:58:10.083164 1326 raft_consensus.cc:360] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.083379 1326 raft_consensus.cc:386] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.083457 1326 raft_consensus.cc:748] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.083688 1326 consensus_queue.cc:261] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.087087 1077 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0bbe34e7bb77494a9c9a1e932af6adb8" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.087296 1077 raft_consensus.cc:3096] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.088835 1344 consensus_queue.cc:1049] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.089244 737 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:39006 (request call id 7) took 1340 ms. Trace:
I20261118 14:58:10.089361 737 rpcz_store.cc:276] 1118 14:58:08.748417 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:08.750699 (+ 2282us) service_pool.cc:224] Handling call
1118 14:58:09.258165 (+507466us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:09.258540 (+ 375us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:10.082050 (+823510us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:10.088988 (+ 6938us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 14:58:10.089100 (+ 112us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 14:58:10.089208 (+ 108us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:10.083149 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 14:58:10.083283 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
1118 14:58:10.083290 (+ 7us) write_op.cc:435] Acquired schema lock
1118 14:58:10.083294 (+ 4us) tablet.cc:670] Decoding operations
1118 14:58:10.083407 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
1118 14:58:10.083422 (+ 15us) write_op.cc:641] Partition lock acquired for write op
1118 14:58:10.083428 (+ 6us) tablet.cc:693] Acquiring locks for 2 operations
1118 14:58:10.083512 (+ 84us) tablet.cc:709] Row locks acquired
1118 14:58:10.083515 (+ 3us) write_op.cc:260] PREPARE: finished
1118 14:58:10.083551 (+ 36us) write_op.cc:270] Start()
1118 14:58:10.083586 (+ 35us) write_op.cc:276] Timestamp: P: 1795013890083501 usec, L: 0
1118 14:58:10.083591 (+ 5us) op_driver.cc:350] REPLICATION: starting
1118 14:58:10.083776 (+ 185us) log.cc:844] Serialized 432 byte log entry
1118 14:58:10.087405 (+ 3629us) op_driver.cc:466] REPLICATION: finished
1118 14:58:10.087742 (+ 337us) write_op.cc:301] APPLY: starting
1118 14:58:10.087759 (+ 17us) tablet.cc:1402] starting BulkCheckPresence
1118 14:58:10.087803 (+ 44us) tablet.cc:1405] finished BulkCheckPresence
1118 14:58:10.087834 (+ 31us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 14:58:10.087962 (+ 128us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 14:58:10.087994 (+ 32us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 14:58:10.088001 (+ 7us) write_op.cc:312] APPLY: finished
1118 14:58:10.088074 (+ 73us) log.cc:844] Serialized 28 byte log entry
1118 14:58:10.088134 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:10.088154 (+ 20us) write_op.cc:454] Released schema lock
1118 14:58:10.088160 (+ 6us) write_op.cc:341] FINISH: Updating metrics
1118 14:58:10.088225 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:10.088229 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":666212,"Hive Metastore.run_cpu_time_us":246,"Hive Metastore.run_wall_time_us":157020,"child_traces":[["op",{"apply.queue_time_us":277,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":481,"num_ops":2,"prepare.queue_time_us":257,"prepare.run_cpu_time_us":751,"prepare.run_wall_time_us":752,"raft.queue_time_us":1576,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":248,"replication_time_us":3785}]]}
I20261118 14:58:10.090230 1077 raft_consensus.cc:2490] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.091380 737 catalog_manager.cc:5840] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "c01f1b88968f4905ba5a401a81ade9f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: HEALTHY } } }
I20261118 14:58:10.096031 1326 ts_tablet_manager.cc:1480] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.013s user 0.000s sys 0.002s
I20261118 14:58:10.098723 1325 tablet_bootstrap.cc:492] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:10.098888 1325 ts_tablet_manager.cc:1449] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.001s
I20261118 14:58:10.099573 1325 raft_consensus.cc:360] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.099761 1325 raft_consensus.cc:386] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.099817 1325 raft_consensus.cc:748] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.100076 1325 consensus_queue.cc:261] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.100507 1325 ts_tablet_manager.cc:1480] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20261118 14:58:10.101836 1384 consensus_queue.cc:1049] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.105970 1327 tablet_bootstrap.cc:492] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:10.106158 1327 ts_tablet_manager.cc:1449] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.058s user 0.004s sys 0.002s
I20261118 14:58:10.107324 1384 consensus_queue.cc:1049] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.107642 1327 raft_consensus.cc:360] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.109066 1327 raft_consensus.cc:386] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.109421 1327 raft_consensus.cc:748] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.109825 1327 consensus_queue.cc:261] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.109826 737 catalog_manager.cc:5840] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c4fe5e44448d98db1eed6329d150d (127.0.72.130). New cstate: current_term: 1 leader_uuid: "245c4fe5e44448d98db1eed6329d150d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: UNKNOWN } } }
I20261118 14:58:10.110230 1344 consensus_queue.cc:1049] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.110323 1327 ts_tablet_manager.cc:1480] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20261118 14:58:10.120935 737 catalog_manager.cc:5840] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129). New cstate: current_term: 1 leader_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: UNKNOWN } } }
I20261118 14:58:10.128517 1191 tablet_service.cc:1500] Processing CreateTablet for tablet 92fab0337a8e4ffeb604589ac588b0af (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.129032 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92fab0337a8e4ffeb604589ac588b0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.141521 1384 raft_consensus.cc:499] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.141690 1384 raft_consensus.cc:521] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.142338 1384 leader_election.cc:290] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.142977 1212 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "91a4a7f508b44f809c8071413df8d648" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:10.143256 1212 raft_consensus.cc:2490] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.143824 874 leader_election.cc:304] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.144160 1384 raft_consensus.cc:2826] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.144302 1384 raft_consensus.cc:499] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.144400 1384 raft_consensus.cc:3096] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.146867 1056 tablet_service.cc:1500] Processing CreateTablet for tablet 92fab0337a8e4ffeb604589ac588b0af (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.147207 1384 raft_consensus.cc:521] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.148595 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92fab0337a8e4ffeb604589ac588b0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.149044 1212 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "91a4a7f508b44f809c8071413df8d648" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:10.149273 1212 raft_consensus.cc:3096] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.150233 1327 tablet_bootstrap.cc:492] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:10.150578 1384 leader_election.cc:290] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.152209 1327 tablet_bootstrap.cc:654] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.152628 1212 raft_consensus.cc:2490] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.153252 874 leader_election.cc:304] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.153648 1384 raft_consensus.cc:2826] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.155128 1384 raft_consensus.cc:705] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:10.155457 1384 consensus_queue.cc:238] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.156051 1077 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "91a4a7f508b44f809c8071413df8d648" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.156279 1077 raft_consensus.cc:2490] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.156625 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "91a4a7f508b44f809c8071413df8d648" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.156795 1075 raft_consensus.cc:3096] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.156828 918 tablet_service.cc:1500] Processing CreateTablet for tablet 92fab0337a8e4ffeb604589ac588b0af (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.157202 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92fab0337a8e4ffeb604589ac588b0af. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.159910 1075 raft_consensus.cc:2490] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.163990 1325 tablet_bootstrap.cc:492] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:10.165933 1325 tablet_bootstrap.cc:654] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.167960 1325 tablet_bootstrap.cc:492] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:10.168123 1325 ts_tablet_manager.cc:1449] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20261118 14:58:10.169070 1325 raft_consensus.cc:360] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.169365 1325 raft_consensus.cc:386] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.169514 1325 raft_consensus.cc:748] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.169797 1325 consensus_queue.cc:261] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.171682 1325 ts_tablet_manager.cc:1480] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20261118 14:58:10.174063 1384 consensus_queue.cc:1049] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.182674 1326 tablet_bootstrap.cc:492] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:10.182590 1387 consensus_queue.cc:1049] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.184636 1326 tablet_bootstrap.cc:654] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.195142 737 catalog_manager.cc:5840] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129). New cstate: current_term: 1 leader_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: UNKNOWN } } }
I20261118 14:58:10.213923 734 catalog_manager.cc:2215] created HMS catalog entry for table [id=91e39a36391d48d99bab0efaef13cdfe]
I20261118 14:58:10.219707 734 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:39006 (request call id 10) took 1467 ms. Trace:
I20261118 14:58:10.219830 734 rpcz_store.cc:276] 1118 14:58:08.751766 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:08.751826 (+ 60us) service_pool.cc:224] Handling call
1118 14:58:09.258817 (+506991us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:09.259131 (+ 314us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:10.213877 (+954746us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:10.219443 (+ 5566us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 14:58:10.219566 (+ 123us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 14:58:10.219672 (+ 106us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:10.214687 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 14:58:10.214818 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
1118 14:58:10.214824 (+ 6us) write_op.cc:435] Acquired schema lock
1118 14:58:10.214827 (+ 3us) tablet.cc:670] Decoding operations
1118 14:58:10.214921 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
1118 14:58:10.214935 (+ 14us) write_op.cc:641] Partition lock acquired for write op
1118 14:58:10.214940 (+ 5us) tablet.cc:693] Acquiring locks for 2 operations
1118 14:58:10.215020 (+ 80us) tablet.cc:709] Row locks acquired
1118 14:58:10.215023 (+ 3us) write_op.cc:260] PREPARE: finished
1118 14:58:10.215055 (+ 32us) write_op.cc:270] Start()
1118 14:58:10.215084 (+ 29us) write_op.cc:276] Timestamp: P: 1795013890215041 usec, L: 0
1118 14:58:10.215088 (+ 4us) op_driver.cc:350] REPLICATION: starting
1118 14:58:10.215251 (+ 163us) log.cc:844] Serialized 432 byte log entry
1118 14:58:10.218829 (+ 3578us) op_driver.cc:466] REPLICATION: finished
1118 14:58:10.218955 (+ 126us) write_op.cc:301] APPLY: starting
1118 14:58:10.218969 (+ 14us) tablet.cc:1402] starting BulkCheckPresence
1118 14:58:10.219002 (+ 33us) tablet.cc:1405] finished BulkCheckPresence
1118 14:58:10.219032 (+ 30us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 14:58:10.219124 (+ 92us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 14:58:10.219152 (+ 28us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 14:58:10.219160 (+ 8us) write_op.cc:312] APPLY: finished
1118 14:58:10.219218 (+ 58us) log.cc:844] Serialized 28 byte log entry
1118 14:58:10.219270 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:10.219287 (+ 17us) write_op.cc:454] Released schema lock
1118 14:58:10.219292 (+ 5us) write_op.cc:341] FINISH: Updating metrics
1118 14:58:10.219352 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:10.219356 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":822731,"Hive Metastore.run_cpu_time_us":269,"Hive Metastore.run_wall_time_us":131777,"spinlock_wait_cycles":90880,"child_traces":[["op",{"apply.queue_time_us":88,"apply.run_cpu_time_us":386,"apply.run_wall_time_us":386,"num_ops":2,"prepare.queue_time_us":40,"prepare.run_cpu_time_us":655,"prepare.run_wall_time_us":656,"raft.queue_time_us":56,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":3717}]]}
I20261118 14:58:10.222180 734 master_service.cc:965] ReplaceTablet: received request to replace tablet 0bbe34e7bb77494a9c9a1e932af6adb8 from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:10.222750 1326 tablet_bootstrap.cc:492] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:10.222911 1326 ts_tablet_manager.cc:1449] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.005s
I20261118 14:58:10.223668 1326 raft_consensus.cc:360] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.223899 1326 raft_consensus.cc:386] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.223995 1326 raft_consensus.cc:748] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.224295 1326 consensus_queue.cc:261] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.225013 1326 ts_tablet_manager.cc:1480] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:10.232589 734 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet 0bbe34e7bb77494a9c9a1e932af6adb8
I20261118 14:58:10.234068 734 catalog_manager.cc:6936] ReplaceTablet: tablet 0bbe34e7bb77494a9c9a1e932af6adb8 deleted and replaced by tablet cdc9a8de0308404fb2fa23b70984f12b
I20261118 14:58:10.237293 1327 tablet_bootstrap.cc:492] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:10.237452 1327 ts_tablet_manager.cc:1449] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.087s user 0.020s sys 0.019s
I20261118 14:58:10.238237 1327 raft_consensus.cc:360] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.238458 1327 raft_consensus.cc:386] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.238539 1327 raft_consensus.cc:748] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.238768 1327 consensus_queue.cc:261] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.239432 1327 ts_tablet_manager.cc:1480] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:10.241796 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 0bbe34e7bb77494a9c9a1e932af6adb8 with delete_type TABLET_DATA_DELETED (replaced by tablet cdc9a8de0308404fb2fa23b70984f12b) from {username='slave'} at 127.0.0.1:49472
I20261118 14:58:10.242069 1334 tablet_replica.cc:333] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:10.242269 1334 raft_consensus.cc:2261] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:10.242450 1334 pending_rounds.cc:70] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Trying to abort 1 pending ops.
I20261118 14:58:10.242507 1334 pending_rounds.cc:77] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7352376893768261632 op_type: NO_OP noop_request { }
I20261118 14:58:10.242614 1334 raft_consensus.cc:2917] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20261118 14:58:10.242724 1334 raft_consensus.cc:2290] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
W20261118 14:58:10.244856 875 consensus_peers.cc:597] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:10.246557 737 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_155ca12a699d4393b30979580a04864a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:10.251578 918 tablet_service.cc:1547] Processing DeleteTablet for tablet 0bbe34e7bb77494a9c9a1e932af6adb8 with delete_type TABLET_DATA_DELETED (replaced by tablet cdc9a8de0308404fb2fa23b70984f12b) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:10.251941 1335 tablet_replica.cc:333] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:10.252168 1335 raft_consensus.cc:2261] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:10.252671 1335 raft_consensus.cc:2290] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.253836 1335 ts_tablet_manager.cc:1962] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.261366 1192 tablet_service.cc:1500] Processing CreateTablet for tablet 75581812cbcb48e5a2ea9d0fb925c03a (DEFAULT_TABLE table=default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.261843 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75581812cbcb48e5a2ea9d0fb925c03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.263120 1056 tablet_service.cc:1500] Processing CreateTablet for tablet 75581812cbcb48e5a2ea9d0fb925c03a (DEFAULT_TABLE table=default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.263566 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75581812cbcb48e5a2ea9d0fb925c03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.266645 1387 raft_consensus.cc:499] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.266881 1387 raft_consensus.cc:521] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.267565 1387 leader_election.cc:290] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.268144 1191 tablet_service.cc:1547] Processing DeleteTablet for tablet 0bbe34e7bb77494a9c9a1e932af6adb8 with delete_type TABLET_DATA_DELETED (replaced by tablet cdc9a8de0308404fb2fa23b70984f12b) from {username='slave'} at 127.0.0.1:60862
I20261118 14:58:10.268405 1333 tablet_replica.cc:333] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:10.268620 1333 raft_consensus.cc:2261] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:10.268656 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a698428dd3b14661be8e17bac5fd56f1" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:10.268822 1333 raft_consensus.cc:2290] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.268898 1210 raft_consensus.cc:2490] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.269378 874 leader_election.cc:304] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.269791 1387 raft_consensus.cc:2826] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.269881 1387 raft_consensus.cc:499] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.269914 1333 ts_tablet_manager.cc:1962] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.269971 1387 raft_consensus.cc:3096] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.272761 1387 raft_consensus.cc:521] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.275920 1333 ts_tablet_manager.cc:1975] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:10.276063 1333 log.cc:1199] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/0bbe34e7bb77494a9c9a1e932af6adb8
I20261118 14:58:10.276443 1333 ts_tablet_manager.cc:1996] T 0bbe34e7bb77494a9c9a1e932af6adb8 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:10.277556 1335 ts_tablet_manager.cc:1975] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:10.277690 1335 log.cc:1199] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/0bbe34e7bb77494a9c9a1e932af6adb8
I20261118 14:58:10.278059 1335 ts_tablet_manager.cc:1996] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:10.278015 729 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 0bbe34e7bb77494a9c9a1e932af6adb8 (table default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:10.279626 1334 ts_tablet_manager.cc:1962] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.284210 1326 tablet_bootstrap.cc:492] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:10.285041 1387 leader_election.cc:290] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.286041 1326 tablet_bootstrap.cc:654] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.286718 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a698428dd3b14661be8e17bac5fd56f1" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:10.287711 1211 raft_consensus.cc:3096] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.289072 1344 raft_consensus.cc:499] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.289248 1344 raft_consensus.cc:521] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.289362 1327 tablet_bootstrap.cc:492] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:10.289324 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a698428dd3b14661be8e17bac5fd56f1" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.289544 1076 raft_consensus.cc:2490] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.289955 1344 leader_election.cc:290] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.290767 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "92fab0337a8e4ffeb604589ac588b0af" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:10.290969 938 raft_consensus.cc:2490] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:10.291002 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "92fab0337a8e4ffeb604589ac588b0af" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:10.291162 1327 tablet_bootstrap.cc:654] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.291190 1210 raft_consensus.cc:2490] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:10.291668 1010 leader_election.cc:304] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:10.291950 1344 raft_consensus.cc:2826] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.292068 1344 raft_consensus.cc:499] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.292174 1344 raft_consensus.cc:3096] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.292654 727 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 0bbe34e7bb77494a9c9a1e932af6adb8 (table default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:10.293551 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a698428dd3b14661be8e17bac5fd56f1" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.293759 1076 raft_consensus.cc:3096] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.294334 918 tablet_service.cc:1500] Processing CreateTablet for tablet 75581812cbcb48e5a2ea9d0fb925c03a (DEFAULT_TABLE table=default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.294783 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75581812cbcb48e5a2ea9d0fb925c03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.296011 1344 raft_consensus.cc:521] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.296471 1076 raft_consensus.cc:2490] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.297576 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "92fab0337a8e4ffeb604589ac588b0af" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:10.297745 938 raft_consensus.cc:3096] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.298775 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "92fab0337a8e4ffeb604589ac588b0af" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:10.298943 1210 raft_consensus.cc:3096] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.298987 875 leader_election.cc:304] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:10.299820 1387 raft_consensus.cc:2826] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.300699 938 raft_consensus.cc:2490] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:10.301146 1010 leader_election.cc:304] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:10.301263 1387 raft_consensus.cc:705] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:10.301424 1331 raft_consensus.cc:2826] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.301535 1331 raft_consensus.cc:705] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:10.301537 1387 consensus_queue.cc:238] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.301663 1210 raft_consensus.cc:2490] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:10.301776 1331 consensus_queue.cc:238] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.302836 1344 leader_election.cc:290] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.305665 1334 ts_tablet_manager.cc:1975] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:10.305876 1334 log.cc:1199] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/0bbe34e7bb77494a9c9a1e932af6adb8
I20261118 14:58:10.306331 1334 ts_tablet_manager.cc:1996] T 0bbe34e7bb77494a9c9a1e932af6adb8 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:10.307546 730 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 0bbe34e7bb77494a9c9a1e932af6adb8 (table default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:10.309409 1211 raft_consensus.cc:2490] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.309942 1325 tablet_bootstrap.cc:492] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:10.311007 1326 tablet_bootstrap.cc:492] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:10.311187 1326 ts_tablet_manager.cc:1449] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.004s
I20261118 14:58:10.311875 1326 raft_consensus.cc:360] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.311985 1325 tablet_bootstrap.cc:654] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.312067 1326 raft_consensus.cc:386] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.312127 1326 raft_consensus.cc:748] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.314109 1326 consensus_queue.cc:261] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.315378 1326 ts_tablet_manager.cc:1480] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20261118 14:58:10.316780 1327 tablet_bootstrap.cc:492] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:10.317032 1327 ts_tablet_manager.cc:1449] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.000s
I20261118 14:58:10.317902 1327 raft_consensus.cc:360] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.318324 1327 raft_consensus.cc:386] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.318470 1327 raft_consensus.cc:748] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.318796 1327 consensus_queue.cc:261] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.320195 1192 tablet_service.cc:1500] Processing CreateTablet for tablet cdc9a8de0308404fb2fa23b70984f12b (DEFAULT_TABLE table=default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.321452 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc9a8de0308404fb2fa23b70984f12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.319458 1327 ts_tablet_manager.cc:1480] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261118 14:58:10.324225 919 tablet_service.cc:1547] Processing DeleteTablet for tablet 0bbe34e7bb77494a9c9a1e932af6adb8 with delete_type TABLET_DATA_DELETED (replaced by tablet cdc9a8de0308404fb2fa23b70984f12b) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:10.326056 734 master_service.cc:965] ReplaceTablet: received request to replace tablet a698428dd3b14661be8e17bac5fd56f1 from {username='slave'} at 127.0.0.1:39006
W20261118 14:58:10.326660 727 catalog_manager.cc:5032] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet 0bbe34e7bb77494a9c9a1e932af6adb8 because the tablet was not found. No further retry: Not found: Tablet not found: 0bbe34e7bb77494a9c9a1e932af6adb8
I20261118 14:58:10.327060 1345 consensus_queue.cc:1049] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.328795 1056 tablet_service.cc:1500] Processing CreateTablet for tablet cdc9a8de0308404fb2fa23b70984f12b (DEFAULT_TABLE table=default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.330806 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc9a8de0308404fb2fa23b70984f12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.334295 1327 tablet_bootstrap.cc:492] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:10.336050 1327 tablet_bootstrap.cc:654] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.338032 1331 consensus_queue.cc:1049] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.340291 734 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet a698428dd3b14661be8e17bac5fd56f1
I20261118 14:58:10.341153 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet a698428dd3b14661be8e17bac5fd56f1 with delete_type TABLET_DATA_DELETED (replaced by tablet cf7800d54792490faa3860de0ecb7a3a) from {username='slave'} at 127.0.0.1:49472
I20261118 14:58:10.341425 1334 tablet_replica.cc:333] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:10.341606 1334 raft_consensus.cc:2261] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:10.341782 1334 raft_consensus.cc:2290] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.342314 1334 ts_tablet_manager.cc:1962] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.348057 1192 tablet_service.cc:1547] Processing DeleteTablet for tablet a698428dd3b14661be8e17bac5fd56f1 with delete_type TABLET_DATA_DELETED (replaced by tablet cf7800d54792490faa3860de0ecb7a3a) from {username='slave'} at 127.0.0.1:60862
I20261118 14:58:10.348901 734 catalog_manager.cc:6936] ReplaceTablet: tablet a698428dd3b14661be8e17bac5fd56f1 deleted and replaced by tablet cf7800d54792490faa3860de0ecb7a3a
I20261118 14:58:10.350701 1333 tablet_replica.cc:333] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:10.350884 1333 raft_consensus.cc:2261] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:10.351044 1333 raft_consensus.cc:2290] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.351508 1333 ts_tablet_manager.cc:1962] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.351950 1334 ts_tablet_manager.cc:1975] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:10.352065 1334 log.cc:1199] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/a698428dd3b14661be8e17bac5fd56f1
I20261118 14:58:10.352394 1334 ts_tablet_manager.cc:1996] T a698428dd3b14661be8e17bac5fd56f1 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:10.353317 730 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet a698428dd3b14661be8e17bac5fd56f1 (table default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]) successfully deleted
I20261118 14:58:10.355852 1376 raft_consensus.cc:499] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.356001 1376 raft_consensus.cc:521] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.356741 1376 leader_election.cc:290] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:10.358112 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e47d7d10a5c741ecb30ec701c76b27a3" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.358314 1076 raft_consensus.cc:2490] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:10.360628 1191 tablet_service.cc:1500] Processing CreateTablet for tablet cf7800d54792490faa3860de0ecb7a3a (DEFAULT_TABLE table=default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.361023 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf7800d54792490faa3860de0ecb7a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.361429 1333 ts_tablet_manager.cc:1975] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:10.361558 1333 log.cc:1199] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/a698428dd3b14661be8e17bac5fd56f1
I20261118 14:58:10.361897 1333 ts_tablet_manager.cc:1996] T a698428dd3b14661be8e17bac5fd56f1 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:10.362679 1148 leader_election.cc:304] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.362964 1376 raft_consensus.cc:2826] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.363047 1376 raft_consensus.cc:499] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.363138 1376 raft_consensus.cc:3096] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.365888 1376 raft_consensus.cc:521] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.366611 1057 tablet_service.cc:1500] Processing CreateTablet for tablet cf7800d54792490faa3860de0ecb7a3a (DEFAULT_TABLE table=default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.366904 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e47d7d10a5c741ecb30ec701c76b27a3" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.366982 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf7800d54792490faa3860de0ecb7a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.367087 1076 raft_consensus.cc:3096] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.370105 1076 raft_consensus.cc:2490] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:10.370646 1148 leader_election.cc:304] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.370941 1369 raft_consensus.cc:2826] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.371098 1369 raft_consensus.cc:705] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:10.371418 1369 consensus_queue.cc:238] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.375921 1326 tablet_bootstrap.cc:492] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:10.376742 1376 leader_election.cc:290] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:10.376570 734 catalog_manager.cc:5840] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c4fe5e44448d98db1eed6329d150d (127.0.72.130). New cstate: current_term: 1 leader_uuid: "245c4fe5e44448d98db1eed6329d150d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: UNKNOWN } } }
I20261118 14:58:10.377555 1326 tablet_bootstrap.cc:654] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.391196 729 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet a698428dd3b14661be8e17bac5fd56f1 (table default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]) successfully deleted
I20261118 14:58:10.392884 734 master_service.cc:965] ReplaceTablet: received request to replace tablet 07d0bc6b6bb6484892091f81ecdc505a from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:10.396108 1332 raft_consensus.cc:499] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.396486 919 tablet_service.cc:1547] Processing DeleteTablet for tablet a698428dd3b14661be8e17bac5fd56f1 with delete_type TABLET_DATA_DELETED (replaced by tablet cf7800d54792490faa3860de0ecb7a3a) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:10.397670 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e47d7d10a5c741ecb30ec701c76b27a3" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:10.398185 939 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e47d7d10a5c741ecb30ec701c76b27a3" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:10.398677 1332 raft_consensus.cc:521] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.399397 1332 leader_election.cc:290] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.399823 937 raft_consensus.cc:3096] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.400686 1326 tablet_bootstrap.cc:492] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:10.400877 1326 ts_tablet_manager.cc:1449] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.006s
I20261118 14:58:10.401659 1326 raft_consensus.cc:360] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.401829 1326 raft_consensus.cc:386] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.401901 1326 raft_consensus.cc:748] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.402159 1326 consensus_queue.cc:261] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.402661 1326 ts_tablet_manager.cc:1480] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:10.402856 919 tablet_service.cc:1500] Processing CreateTablet for tablet cf7800d54792490faa3860de0ecb7a3a (DEFAULT_TABLE table=default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.403276 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf7800d54792490faa3860de0ecb7a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.403923 734 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet 07d0bc6b6bb6484892091f81ecdc505a
I20261118 14:58:10.407101 917 tablet_service.cc:1547] Processing DeleteTablet for tablet 07d0bc6b6bb6484892091f81ecdc505a with delete_type TABLET_DATA_DELETED (replaced by tablet bbf3d70de6b143e88c9dfc83f7648e8f) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:10.407590 1335 tablet_replica.cc:333] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:10.407944 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e47d7d10a5c741ecb30ec701c76b27a3" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:10.408064 937 raft_consensus.cc:2490] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:10.408507 1326 tablet_bootstrap.cc:492] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:10.409369 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e47d7d10a5c741ecb30ec701c76b27a3" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.409749 1076 raft_consensus.cc:2415] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c53fe91804746e8bfcf743e0ab8ba6f in current term 1: Already voted for candidate c01f1b88968f4905ba5a401a81ade9f7 in this term.
I20261118 14:58:10.410372 1326 tablet_bootstrap.cc:654] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.410262 875 leader_election.cc:304] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f; no voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7
I20261118 14:58:10.410766 1332 raft_consensus.cc:2771] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 14:58:10.411785 1325 tablet_bootstrap.cc:492] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:10.412001 1325 ts_tablet_manager.cc:1449] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.102s user 0.007s sys 0.003s
I20261118 14:58:10.412261 1192 tablet_service.cc:1547] Processing DeleteTablet for tablet 07d0bc6b6bb6484892091f81ecdc505a with delete_type TABLET_DATA_DELETED (replaced by tablet bbf3d70de6b143e88c9dfc83f7648e8f) from {username='slave'} at 127.0.0.1:60862
I20261118 14:58:10.412845 1333 tablet_replica.cc:333] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:10.412809 1325 raft_consensus.cc:360] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.413012 1325 raft_consensus.cc:386] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.413092 1325 raft_consensus.cc:748] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.413214 1333 raft_consensus.cc:2261] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:10.413367 1325 consensus_queue.cc:261] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.413826 1333 raft_consensus.cc:2290] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.414934 1325 ts_tablet_manager.cc:1480] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20261118 14:58:10.416196 1325 tablet_bootstrap.cc:492] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:10.417830 1325 tablet_bootstrap.cc:654] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.420085 918 tablet_service.cc:1500] Processing CreateTablet for tablet cdc9a8de0308404fb2fa23b70984f12b (DEFAULT_TABLE table=default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.423377 1192 tablet_service.cc:1500] Processing CreateTablet for tablet bbf3d70de6b143e88c9dfc83f7648e8f (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.424041 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf3d70de6b143e88c9dfc83f7648e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.424387 1057 tablet_service.cc:1500] Processing CreateTablet for tablet bbf3d70de6b143e88c9dfc83f7648e8f (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.424748 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf3d70de6b143e88c9dfc83f7648e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.430730 734 catalog_manager.cc:6936] ReplaceTablet: tablet 07d0bc6b6bb6484892091f81ecdc505a deleted and replaced by tablet bbf3d70de6b143e88c9dfc83f7648e8f
I20261118 14:58:10.431932 1327 tablet_bootstrap.cc:492] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:10.432134 1327 ts_tablet_manager.cc:1449] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.098s user 0.005s sys 0.002s
I20261118 14:58:10.432524 734 master_service.cc:965] ReplaceTablet: received request to replace tablet dc2c699d9f4746c3905667c67d20e987 from {username='slave'} at 127.0.0.1:39006
I20261118 14:58:10.432880 1326 tablet_bootstrap.cc:492] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:10.433035 1326 ts_tablet_manager.cc:1449] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.000s
I20261118 14:58:10.433279 734 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:10.433527 734 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:10.433795 1326 raft_consensus.cc:360] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.433940 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 07d0bc6b6bb6484892091f81ecdc505a with delete_type TABLET_DATA_DELETED (replaced by tablet bbf3d70de6b143e88c9dfc83f7648e8f) from {username='slave'} at 127.0.0.1:49472
I20261118 14:58:10.433984 1326 raft_consensus.cc:386] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.434048 1326 raft_consensus.cc:748] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.434188 1334 tablet_replica.cc:333] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:10.434405 1334 raft_consensus.cc:2261] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:10.434401 1326 consensus_queue.cc:261] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.434616 1334 raft_consensus.cc:2290] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.434916 1326 ts_tablet_manager.cc:1480] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 14:58:10.435506 1334 ts_tablet_manager.cc:1962] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.439383 1327 raft_consensus.cc:360] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.439661 1327 raft_consensus.cc:386] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.439795 1327 raft_consensus.cc:748] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.440093 1327 consensus_queue.cc:261] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.440765 1327 ts_tablet_manager.cc:1480] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20261118 14:58:10.441072 1327 tablet_bootstrap.cc:492] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:10.423975 919 tablet_service.cc:1500] Processing CreateTablet for tablet bbf3d70de6b143e88c9dfc83f7648e8f (DEFAULT_TABLE table=default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:10.442413 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf3d70de6b143e88c9dfc83f7648e8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.442746 1326 tablet_bootstrap.cc:492] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:10.444042 1334 ts_tablet_manager.cc:1975] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:10.445286 1325 tablet_bootstrap.cc:492] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:10.445487 1325 ts_tablet_manager.cc:1449] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.003s
I20261118 14:58:10.446298 1376 raft_consensus.cc:499] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.446465 1376 raft_consensus.cc:521] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.448318 1376 leader_election.cc:290] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:10.447537 1335 raft_consensus.cc:2261] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:10.449258 1335 pending_rounds.cc:70] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Trying to abort 1 pending ops.
I20261118 14:58:10.449414 1335 pending_rounds.cc:77] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7352376894678007808 op_type: NO_OP noop_request { }
I20261118 14:58:10.449548 1335 raft_consensus.cc:2917] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20261118 14:58:10.449648 1335 raft_consensus.cc:2290] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.447559 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "75581812cbcb48e5a2ea9d0fb925c03a" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:10.449944 938 raft_consensus.cc:2490] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:10.450419 1145 leader_election.cc:304] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.450747 1376 raft_consensus.cc:2826] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.450838 1376 raft_consensus.cc:499] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.450939 1376 raft_consensus.cc:3096] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.451478 1325 raft_consensus.cc:360] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.444388 1326 tablet_bootstrap.cc:654] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.451836 1334 log.cc:1199] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/07d0bc6b6bb6484892091f81ecdc505a
I20261118 14:58:10.452335 1334 ts_tablet_manager.cc:1996] T 07d0bc6b6bb6484892091f81ecdc505a P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:10.452458 1325 raft_consensus.cc:386] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.452596 1325 raft_consensus.cc:748] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.449458 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "75581812cbcb48e5a2ea9d0fb925c03a" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.452847 1076 raft_consensus.cc:2490] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:10.454013 1325 consensus_queue.cc:261] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.454092 730 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 07d0bc6b6bb6484892091f81ecdc505a (table default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:10.454574 1376 raft_consensus.cc:521] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.455551 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "75581812cbcb48e5a2ea9d0fb925c03a" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:10.455755 938 raft_consensus.cc:3096] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.455998 1325 ts_tablet_manager.cc:1480] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20261118 14:58:10.457682 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc9a8de0308404fb2fa23b70984f12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:10.458598 938 raft_consensus.cc:2490] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:10.459179 1145 leader_election.cc:304] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.459463 1337 raft_consensus.cc:2826] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.459580 1337 raft_consensus.cc:705] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:10.459833 1337 consensus_queue.cc:238] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.463925 1368 consensus_queue.cc:1049] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.464870 1325 tablet_bootstrap.cc:492] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:10.466742 1325 tablet_bootstrap.cc:654] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.467950 1376 leader_election.cc:290] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:10.468451 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "75581812cbcb48e5a2ea9d0fb925c03a" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.468505 1333 ts_tablet_manager.cc:1962] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.468654 1075 raft_consensus.cc:3096] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.471422 1075 raft_consensus.cc:2490] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:10.474989 1335 ts_tablet_manager.cc:1962] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.475997 1326 tablet_bootstrap.cc:492] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:10.476136 1326 ts_tablet_manager.cc:1449] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.033s user 0.005s sys 0.004s
I20261118 14:58:10.476732 1368 consensus_queue.cc:1049] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.478611 1327 tablet_bootstrap.cc:654] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.479308 1325 tablet_bootstrap.cc:492] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:10.479527 1325 ts_tablet_manager.cc:1449] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20261118 14:58:10.480264 1325 raft_consensus.cc:360] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.480460 1325 raft_consensus.cc:386] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.480531 1325 raft_consensus.cc:748] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.481622 1333 ts_tablet_manager.cc:1975] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:10.481746 1333 log.cc:1199] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/07d0bc6b6bb6484892091f81ecdc505a
I20261118 14:58:10.482093 1333 ts_tablet_manager.cc:1996] T 07d0bc6b6bb6484892091f81ecdc505a P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:10.483147 1335 ts_tablet_manager.cc:1975] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:10.483286 1335 log.cc:1199] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/a698428dd3b14661be8e17bac5fd56f1
I20261118 14:58:10.483651 1335 ts_tablet_manager.cc:1996] T a698428dd3b14661be8e17bac5fd56f1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:10.484558 1335 tablet_replica.cc:333] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:10.484588 729 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 07d0bc6b6bb6484892091f81ecdc505a (table default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:10.484743 1335 raft_consensus.cc:2261] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:10.484913 1335 raft_consensus.cc:2290] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:10.485138 727 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet a698428dd3b14661be8e17bac5fd56f1 (table default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]) successfully deleted
I20261118 14:58:10.485399 1326 raft_consensus.cc:360] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.485476 1325 consensus_queue.cc:261] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.485584 1326 raft_consensus.cc:386] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.485664 1326 raft_consensus.cc:748] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.485918 1326 consensus_queue.cc:261] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.486141 1325 ts_tablet_manager.cc:1480] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20261118 14:58:10.487907 1325 tablet_bootstrap.cc:492] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:10.488855 1326 ts_tablet_manager.cc:1480] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20261118 14:58:10.489635 1325 tablet_bootstrap.cc:654] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.491724 1335 ts_tablet_manager.cc:1962] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:10.499116 1337 consensus_queue.cc:1049] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.499483 1369 consensus_queue.cc:1049] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.503113 1335 ts_tablet_manager.cc:1975] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:10.503228 1335 log.cc:1199] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/07d0bc6b6bb6484892091f81ecdc505a
I20261118 14:58:10.503571 1335 ts_tablet_manager.cc:1996] T 07d0bc6b6bb6484892091f81ecdc505a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:10.504585 727 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 07d0bc6b6bb6484892091f81ecdc505a (table default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:10.527297 1325 tablet_bootstrap.cc:492] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:10.527482 1325 ts_tablet_manager.cc:1449] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.002s
I20261118 14:58:10.529035 1325 raft_consensus.cc:360] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.529227 1325 raft_consensus.cc:386] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.529328 1325 raft_consensus.cc:748] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.529568 1325 consensus_queue.cc:261] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.530786 1327 tablet_bootstrap.cc:492] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:10.530951 1327 ts_tablet_manager.cc:1449] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.090s user 0.006s sys 0.040s
I20261118 14:58:10.531656 1327 raft_consensus.cc:360] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.531843 1327 raft_consensus.cc:386] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.531912 1327 raft_consensus.cc:748] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.532121 1327 consensus_queue.cc:261] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.532447 1325 ts_tablet_manager.cc:1480] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20261118 14:58:10.533677 1327 ts_tablet_manager.cc:1480] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20261118 14:58:10.534117 1327 tablet_bootstrap.cc:492] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:10.535931 1327 tablet_bootstrap.cc:654] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:10.539428 1327 tablet_bootstrap.cc:492] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:10.539610 1327 ts_tablet_manager.cc:1449] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20261118 14:58:10.540472 1327 raft_consensus.cc:360] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.540726 1327 raft_consensus.cc:386] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:10.540838 1327 raft_consensus.cc:748] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:10.541109 1327 consensus_queue.cc:261] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.541664 1327 ts_tablet_manager.cc:1480] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261118 14:58:10.558670 1387 raft_consensus.cc:499] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.558867 1387 raft_consensus.cc:521] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.559640 1387 leader_election.cc:290] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.560433 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "bbf3d70de6b143e88c9dfc83f7648e8f" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.560683 1076 raft_consensus.cc:2490] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.561055 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "bbf3d70de6b143e88c9dfc83f7648e8f" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:10.561265 1211 raft_consensus.cc:2490] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.561767 874 leader_election.cc:304] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.562480 1387 raft_consensus.cc:2826] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.562611 1387 raft_consensus.cc:499] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.562711 1387 raft_consensus.cc:3096] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.565764 1387 raft_consensus.cc:521] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.566975 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "bbf3d70de6b143e88c9dfc83f7648e8f" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.567212 1076 raft_consensus.cc:3096] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.570118 1076 raft_consensus.cc:2490] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.570703 875 leader_election.cc:304] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:10.571013 1332 raft_consensus.cc:2826] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.571178 1332 raft_consensus.cc:705] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:10.571456 1332 consensus_queue.cc:238] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.574047 1391 raft_consensus.cc:499] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.574200 1391 raft_consensus.cc:521] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.574954 1391 leader_election.cc:290] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.575676 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f4bdcca79fcb4fedaea1fdbe2dd31112" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:10.575897 1211 raft_consensus.cc:2490] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.576426 874 leader_election.cc:304] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.577167 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f4bdcca79fcb4fedaea1fdbe2dd31112" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.577366 1075 raft_consensus.cc:2490] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.578061 1391 raft_consensus.cc:2826] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.578198 1391 raft_consensus.cc:499] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.578296 1391 raft_consensus.cc:3096] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.581346 1391 raft_consensus.cc:521] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.582541 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f4bdcca79fcb4fedaea1fdbe2dd31112" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.582763 1076 raft_consensus.cc:3096] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.585918 1076 raft_consensus.cc:2490] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.586501 875 leader_election.cc:304] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:10.586862 1384 raft_consensus.cc:2826] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.590606 1384 raft_consensus.cc:705] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:10.591020 1384 consensus_queue.cc:238] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.592024 1211 raft_consensus.cc:3096] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.598852 1391 leader_election.cc:290] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.600495 1391 consensus_queue.cc:1049] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20261118 14:58:10.602226 1387 leader_election.cc:290] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.608122 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f4bdcca79fcb4fedaea1fdbe2dd31112" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:10.608341 1211 raft_consensus.cc:3096] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.612171 1211 raft_consensus.cc:2490] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.613166 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "bbf3d70de6b143e88c9dfc83f7648e8f" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:10.614984 1429 consensus_queue.cc:1049] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.616703 1211 raft_consensus.cc:2490] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.631192 1384 consensus_queue.cc:1049] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.645187 1430 consensus_queue.cc:1049] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.875916 1369 raft_consensus.cc:499] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.876157 1369 raft_consensus.cc:521] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.876904 1369 leader_election.cc:290] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:10.877797 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cdc9a8de0308404fb2fa23b70984f12b" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:10.878046 937 raft_consensus.cc:2490] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:10.878769 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cdc9a8de0308404fb2fa23b70984f12b" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.879002 1076 raft_consensus.cc:2490] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:10.879009 1145 leader_election.cc:304] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.879817 1369 raft_consensus.cc:2826] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.879912 1369 raft_consensus.cc:499] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.880004 1369 raft_consensus.cc:3096] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.883484 1369 raft_consensus.cc:521] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.884816 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cdc9a8de0308404fb2fa23b70984f12b" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:10.885047 937 raft_consensus.cc:3096] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.887492 1369 leader_election.cc:290] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:10.888015 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cdc9a8de0308404fb2fa23b70984f12b" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.888244 1076 raft_consensus.cc:3096] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.888392 937 raft_consensus.cc:2490] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:10.888867 1145 leader_election.cc:304] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.889215 1369 raft_consensus.cc:2826] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.889869 1369 raft_consensus.cc:705] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:10.890172 1369 consensus_queue.cc:238] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.890965 1076 raft_consensus.cc:2490] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:10.894667 1369 consensus_queue.cc:1049] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.895009 1386 consensus_queue.cc:1049] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 14:58:10.928263 1355 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:10.928695 1355 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:10.928820 1355 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20261118 14:58:10.934065 1384 raft_consensus.cc:499] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:10.934253 1384 raft_consensus.cc:521] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.934981 1384 leader_election.cc:290] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.935657 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cf7800d54792490faa3860de0ecb7a3a" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:10.935877 1210 raft_consensus.cc:2490] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.935932 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cf7800d54792490faa3860de0ecb7a3a" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:10.936152 1076 raft_consensus.cc:2490] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:10.936393 874 leader_election.cc:304] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:10.936687 1384 raft_consensus.cc:2826] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:10.936800 1384 raft_consensus.cc:499] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:10.936879 1384 raft_consensus.cc:3096] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.939782 1384 raft_consensus.cc:521] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:10.940717 1384 leader_election.cc:290] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:10.940798 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cf7800d54792490faa3860de0ecb7a3a" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:10.940856 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cf7800d54792490faa3860de0ecb7a3a" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:10.940964 1076 raft_consensus.cc:3096] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.941049 1210 raft_consensus.cc:3096] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:10.943077 1076 raft_consensus.cc:2490] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.943542 875 leader_election.cc:304] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:10.943895 1384 raft_consensus.cc:2826] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:10.944406 1210 raft_consensus.cc:2490] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:10.944978 1384 raft_consensus.cc:705] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:10.945397 1384 consensus_queue.cc:238] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
W20261118 14:58:10.945863 1355 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:10.945971 1355 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:10.946058 1355 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:10.946125 1355 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:10.948918 1332 consensus_queue.cc:1049] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.950342 1332 consensus_queue.cc:1049] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:10.982012 1355 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:46707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=42561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:10.984308 1355 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:10.986831 1355 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:10.998293 1448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:11.003943 1449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:11.006819 742 catalog_manager.cc:3543] renamed table default.table_b1a3822c61cb4043b4b9ddeaf29cc405 [id=07fa88e7fcd044fca002cc02a8788a4d] in HMS: new name default.table_537f6e87a8d040a695fda278ff06cbf5
W20261118 14:58:11.033061 1451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:11.033718 1355 server_base.cc:1061] running on GCE node
I20261118 14:58:11.034842 1355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:11.037943 1355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:11.039165 1355 hybrid_clock.cc:648] HybridClock initialized: now 1795013891039094 us; error 66 us; skew 500 ppm
I20261118 14:58:11.039554 1355 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:11.061580 1355 webserver.cc:577] Webserver started at http://127.0.72.189:42561/ using document root <none> and password file <none>
I20261118 14:58:11.062831 1355 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:11.062978 1355 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:11.073194 1355 fs_manager.cc:714] Time spent opening directory manager: real 0.007s user 0.007s sys 0.000s
W20261118 14:58:11.080713 802 catalog_manager.cc:6108] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bbf3d70de6b143e88c9dfc83f7648e8f consensus state has no leader
I20261118 14:58:11.082545 1462 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:11.084619 1355 fs_manager.cc:730] Time spent opening block manager: real 0.010s user 0.002s sys 0.004s
I20261118 14:58:11.084826 1355 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
uuid: "7c9ef96ff074450a879e66787486386d"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:11.085861 1355 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20261118 14:58:11.086447 740 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:11.087561 741 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:11.088687 735 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_9021a49aff8546eca95c57bde20822b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:11.089797 737 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_155ca12a699d4393b30979580a04864a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:11.112895 728 catalog_manager.cc:4847] Async tablet task AlterTable RPC for tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet bbf3d70de6b143e88c9dfc83f7648e8f consensus state has no leader
I20261118 14:58:11.121090 1355 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:11.122146 1355 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:11.122502 1355 kserver.cc:163] Server-wide thread pool size limit: 3276
W20261118 14:58:11.149194 728 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 -> Peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Couldn't send request to peer 7c9ef96ff074450a879e66787486386d. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20261118 14:58:11.151686 1355 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:46707
I20261118 14:58:11.151705 1513 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:46707 every 8 connection(s)
I20261118 14:58:11.153282 1355 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
I20261118 14:58:11.154784 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 1355
I20261118 14:58:11.161777 1514 sys_catalog.cc:263] Verifying existing consensus state
I20261118 14:58:11.165808 1514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap starting.
I20261118 14:58:11.174096 1514 log.cc:826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:11.183156 1514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 14:58:11.183681 1514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap complete.
I20261118 14:58:11.188902 1514 raft_consensus.cc:360] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:11.189555 1514 raft_consensus.cc:748] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9ef96ff074450a879e66787486386d, State: Initialized, Role: FOLLOWER
I20261118 14:58:11.190197 1514 consensus_queue.cc:261] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:11.191073 1518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:11.191358 1518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:11.192176 1514 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:11.207484 743 catalog_manager.cc:3543] renamed table default.table_ab0fd883cbec4e0b8af7febddffe78c5 [id=e26c2363b15444a5882d6ccf7e290f8b] in HMS: new name default.table_2f1acae65de24ae9a7a19515d10acf9c
I20261118 14:58:11.221871 1532 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
W20261118 14:58:11.223894 802 catalog_manager.cc:6108] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f4bdcca79fcb4fedaea1fdbe2dd31112 consensus state has no leader
I20261118 14:58:11.224134 1532 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: loading cluster ID for follower catalog manager: success
I20261118 14:58:11.226003 742 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:39006 (request call id 11) took 2013 ms. Trace:
I20261118 14:58:11.226138 742 rpcz_store.cc:276] 1118 14:58:09.212234 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:09.215458 (+ 3224us) service_pool.cc:224] Handling call
1118 14:58:09.291428 (+ 75970us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 14:58:11.225322 (+1933894us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 14:58:11.225665 (+ 343us) catalog_manager.cc:3766] Apply alter schema
1118 14:58:11.225967 (+ 302us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":964810,"Hive Metastore.run_cpu_time_us":1212,"Hive Metastore.run_wall_time_us":750267}
W20261118 14:58:11.228125 742 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.130:50135 (request call id 18) took 793 ms (client timeout 1000 ms). Trace:
W20261118 14:58:11.228251 742 rpcz_store.cc:269] 1118 14:58:10.434626 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:11.226513 (+791887us) service_pool.cc:224] Handling call
1118 14:58:11.228076 (+ 1563us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2}
I20261118 14:58:11.231365 918 tablet_service.cc:1547] Processing DeleteTablet for tablet a698428dd3b14661be8e17bac5fd56f1 with delete_type TABLET_DATA_DELETED (replaced by tablet cf7800d54792490faa3860de0ecb7a3a) from {username='slave'} at 127.0.0.1:54848
I20261118 14:58:11.233414 1532 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: acquiring CA information for follower catalog manager: success
I20261118 14:58:11.237088 1532 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20261118 14:58:11.238668 742 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.129:47561 (request call id 14) took 790 ms (client timeout 1000 ms). Trace:
W20261118 14:58:11.238802 742 rpcz_store.cc:269] 1118 14:58:10.448036 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:11.228539 (+780503us) service_pool.cc:224] Handling call
1118 14:58:11.238619 (+ 10080us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20261118 14:58:11.242377 742 catalog_manager.cc:5840] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "c01f1b88968f4905ba5a401a81ade9f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: HEALTHY } } }
W20261118 14:58:11.252223 742 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.131:57667 (request call id 17) took 772 ms (client timeout 1000 ms). Trace:
W20261118 14:58:11.252420 742 rpcz_store.cc:269] 1118 14:58:10.479951 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:11.239796 (+759845us) service_pool.cc:224] Handling call
1118 14:58:11.252166 (+ 12370us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:11.244472 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 14:58:11.244626 (+ 154us) write_op.cc:432] Acquiring schema lock in shared mode
1118 14:58:11.244632 (+ 6us) write_op.cc:435] Acquired schema lock
1118 14:58:11.244635 (+ 3us) tablet.cc:670] Decoding operations
1118 14:58:11.244770 (+ 135us) write_op.cc:620] Acquiring the partition lock for write op
1118 14:58:11.244787 (+ 17us) write_op.cc:641] Partition lock acquired for write op
1118 14:58:11.244793 (+ 6us) tablet.cc:693] Acquiring locks for 1 operations
1118 14:58:11.244869 (+ 76us) tablet.cc:709] Row locks acquired
1118 14:58:11.244872 (+ 3us) write_op.cc:260] PREPARE: finished
1118 14:58:11.244914 (+ 42us) write_op.cc:270] Start()
1118 14:58:11.244954 (+ 40us) write_op.cc:276] Timestamp: P: 1795013891244893 usec, L: 0
1118 14:58:11.244958 (+ 4us) op_driver.cc:350] REPLICATION: starting
1118 14:58:11.245231 (+ 273us) log.cc:844] Serialized 529 byte log entry
1118 14:58:11.249204 (+ 3973us) op_driver.cc:466] REPLICATION: finished
1118 14:58:11.249345 (+ 141us) write_op.cc:301] APPLY: starting
1118 14:58:11.249365 (+ 20us) tablet.cc:1402] starting BulkCheckPresence
1118 14:58:11.249414 (+ 49us) tablet.cc:1405] finished BulkCheckPresence
1118 14:58:11.249441 (+ 27us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 14:58:11.249500 (+ 59us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 14:58:11.249534 (+ 34us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 14:58:11.249542 (+ 8us) write_op.cc:312] APPLY: finished
1118 14:58:11.249626 (+ 84us) log.cc:844] Serialized 22 byte log entry
1118 14:58:11.249690 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:11.249711 (+ 21us) write_op.cc:454] Released schema lock
1118 14:58:11.249719 (+ 8us) write_op.cc:341] FINISH: Updating metrics
1118 14:58:11.249785 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:11.249789 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":90,"apply.run_cpu_time_us":436,"apply.run_wall_time_us":436,"num_ops":1,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":887,"prepare.run_wall_time_us":888,"raft.queue_time_us":170,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":275,"replication_time_us":4217}]]}
W20261118 14:58:11.255659 727 catalog_manager.cc:5032] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet a698428dd3b14661be8e17bac5fd56f1 because the tablet was not found. No further retry: Not found: Tablet not found: a698428dd3b14661be8e17bac5fd56f1
I20261118 14:58:11.255147 742 catalog_manager.cc:5840] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "c01f1b88968f4905ba5a401a81ade9f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } }
I20261118 14:58:11.256348 742 catalog_manager.cc:5840] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129). New cstate: current_term: 1 leader_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } }
I20261118 14:58:11.256846 742 catalog_manager.cc:5840] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129). New cstate: current_term: 1 leader_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } }
I20261118 14:58:11.257189 742 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d]). Expected version 1 got 0
I20261118 14:58:11.257493 742 catalog_manager.cc:5840] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "c01f1b88968f4905ba5a401a81ade9f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } }
I20261118 14:58:11.257963 742 catalog_manager.cc:5840] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129). New cstate: current_term: 1 leader_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } }
I20261118 14:58:11.258462 742 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
I20261118 14:58:11.276662 742 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
I20261118 14:58:11.277202 742 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d]). Expected version 1 got 0
I20261118 14:58:11.301757 986 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:11.302337 742 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d]). Expected version 1 got 0
I20261118 14:58:11.302727 742 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
I20261118 14:58:11.306118 742 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
I20261118 14:58:11.306182 1479 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" instance_seqno: 1795013887687621) as {username='slave'} at 127.0.72.129:47587; Asking this server to re-register.
I20261118 14:58:11.309062 742 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_7171b2f0b6ea470393bf9aa28246cf65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:11.310747 986 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:11.313333 1479 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:11.325264 1544 tablet.cc:1759] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:11.348304 1545 tablet.cc:1759] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:11.353694 1546 tablet.cc:1759] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:11.368525 1548 tablet.cc:1759] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:11.374734 1547 tablet.cc:1759] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:11.380121 1549 tablet.cc:1759] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:11.435345 1259 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:11.436952 1124 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:11.439646 1479 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" instance_seqno: 1795013888201158) as {username='slave'} at 127.0.72.130:37397; Asking this server to re-register.
I20261118 14:58:11.439791 1478 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" instance_seqno: 1795013888668865) as {username='slave'} at 127.0.72.131:57249; Asking this server to re-register.
I20261118 14:58:11.440438 1124 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:11.441375 1259 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:11.442755 1479 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:11.443156 1478 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:11.614089 739 catalog_manager.cc:3543] renamed table default.table_59239feaba874d6890fe8e86d57545e2 [id=886aa9bb52444e1186957fd7e610fb28] in HMS: new name default.table_a69dc21b9f4948ffad3423d68bff8f34
I20261118 14:58:11.725131 1552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b90720bc0b34e4994abf4fc6f9e59e1. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:11.725505 1552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:11.770880 1558 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:11.771039 1558 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:11.771726 1558 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:11.773202 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 48 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:11.779477 1489 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 48 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d" is_pre_election: true
I20261118 14:58:11.780405 588 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:11.780730 1558 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:12.148048 1316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:12.148362 1316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:12.205720 740 catalog_manager.cc:2215] created HMS catalog entry for table [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:12.217820 740 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:39006 (request call id 19) took 2806 ms. Trace:
I20261118 14:58:12.217979 740 rpcz_store.cc:276] 1118 14:58:09.411631 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:09.567676 (+156045us) service_pool.cc:224] Handling call
1118 14:58:11.086558 (+1518882us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:11.087074 (+ 516us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:12.205657 (+1118583us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:12.217352 (+ 11695us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 14:58:12.217492 (+ 140us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 14:58:12.217769 (+ 277us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:12.207799 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 14:58:12.207910 (+ 111us) write_op.cc:432] Acquiring schema lock in shared mode
1118 14:58:12.207915 (+ 5us) write_op.cc:435] Acquired schema lock
1118 14:58:12.207917 (+ 2us) tablet.cc:670] Decoding operations
1118 14:58:12.207995 (+ 78us) write_op.cc:620] Acquiring the partition lock for write op
1118 14:58:12.208007 (+ 12us) write_op.cc:641] Partition lock acquired for write op
1118 14:58:12.208011 (+ 4us) tablet.cc:693] Acquiring locks for 2 operations
1118 14:58:12.208076 (+ 65us) tablet.cc:709] Row locks acquired
1118 14:58:12.208078 (+ 2us) write_op.cc:260] PREPARE: finished
1118 14:58:12.208110 (+ 32us) write_op.cc:270] Start()
1118 14:58:12.208141 (+ 31us) write_op.cc:276] Timestamp: P: 1795013892208047 usec, L: 0
1118 14:58:12.208144 (+ 3us) op_driver.cc:350] REPLICATION: starting
1118 14:58:12.208309 (+ 165us) log.cc:844] Serialized 432 byte log entry
1118 14:58:12.212505 (+ 4196us) op_driver.cc:466] REPLICATION: finished
1118 14:58:12.216583 (+ 4078us) write_op.cc:301] APPLY: starting
1118 14:58:12.216602 (+ 19us) tablet.cc:1402] starting BulkCheckPresence
1118 14:58:12.216649 (+ 47us) tablet.cc:1405] finished BulkCheckPresence
1118 14:58:12.216675 (+ 26us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 14:58:12.216808 (+ 133us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 14:58:12.216845 (+ 37us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 14:58:12.216853 (+ 8us) write_op.cc:312] APPLY: finished
1118 14:58:12.216943 (+ 90us) log.cc:844] Serialized 28 byte log entry
1118 14:58:12.217232 (+ 289us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:12.217255 (+ 23us) write_op.cc:454] Released schema lock
1118 14:58:12.217262 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":526842,"Hive Metastore.run_cpu_time_us":365,"Hive Metastore.run_wall_time_us":591516,"child_traces":[["op",{"apply.queue_time_us":3914,"mutex_wait_us":204,"num_ops":2,"prepare.queue_time_us":1355,"prepare.run_cpu_time_us":643,"prepare.run_wall_time_us":642,"raft.queue_time_us":133,"raft.run_cpu_time_us":890,"raft.run_wall_time_us":890,"replication_time_us":4320,"spinlock_wait_cycles":228224,"thread_start_us":5060,"threads_started":2}]]}
I20261118 14:58:12.231017 1057 tablet_service.cc:1500] Processing CreateTablet for tablet dd0cddf6622a469eb598e9740407a9f0 (DEFAULT_TABLE table=default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:12.231199 1192 tablet_service.cc:1500] Processing CreateTablet for tablet dd0cddf6622a469eb598e9740407a9f0 (DEFAULT_TABLE table=default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:12.231446 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd0cddf6622a469eb598e9740407a9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:12.231655 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd0cddf6622a469eb598e9740407a9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:12.235148 918 tablet_service.cc:1500] Processing CreateTablet for tablet dd0cddf6622a469eb598e9740407a9f0 (DEFAULT_TABLE table=default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:12.235584 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd0cddf6622a469eb598e9740407a9f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:12.238463 740 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b] alter complete (version 1)
W20261118 14:58:12.238742 740 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.131:57667 (request call id 19) took 825 ms (client timeout 1000 ms). Trace:
W20261118 14:58:12.238849 740 rpcz_store.cc:269] 1118 14:58:11.413682 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:12.220777 (+807095us) service_pool.cc:224] Handling call
1118 14:58:12.238681 (+ 17904us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:12.224534 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 14:58:12.224651 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
1118 14:58:12.224657 (+ 6us) write_op.cc:435] Acquired schema lock
1118 14:58:12.224659 (+ 2us) tablet.cc:670] Decoding operations
1118 14:58:12.224746 (+ 87us) write_op.cc:620] Acquiring the partition lock for write op
1118 14:58:12.224756 (+ 10us) write_op.cc:641] Partition lock acquired for write op
1118 14:58:12.224761 (+ 5us) tablet.cc:693] Acquiring locks for 1 operations
1118 14:58:12.224820 (+ 59us) tablet.cc:709] Row locks acquired
1118 14:58:12.224822 (+ 2us) write_op.cc:260] PREPARE: finished
1118 14:58:12.224857 (+ 35us) write_op.cc:270] Start()
1118 14:58:12.224888 (+ 31us) write_op.cc:276] Timestamp: P: 1795013892224789 usec, L: 0
1118 14:58:12.224891 (+ 3us) op_driver.cc:350] REPLICATION: starting
1118 14:58:12.225081 (+ 190us) log.cc:844] Serialized 355 byte log entry
1118 14:58:12.229536 (+ 4455us) op_driver.cc:466] REPLICATION: finished
1118 14:58:12.237888 (+ 8352us) write_op.cc:301] APPLY: starting
1118 14:58:12.237906 (+ 18us) tablet.cc:1402] starting BulkCheckPresence
1118 14:58:12.237947 (+ 41us) tablet.cc:1405] finished BulkCheckPresence
1118 14:58:12.237976 (+ 29us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 14:58:12.238038 (+ 62us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 14:58:12.238067 (+ 29us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 14:58:12.238075 (+ 8us) write_op.cc:312] APPLY: finished
1118 14:58:12.238186 (+ 111us) log.cc:844] Serialized 22 byte log entry
1118 14:58:12.238255 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:12.238275 (+ 20us) write_op.cc:454] Released schema lock
1118 14:58:12.238282 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1118 14:58:12.238376 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:12.238380 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":8278,"apply.run_cpu_time_us":468,"apply.run_wall_time_us":469,"num_ops":1,"prepare.queue_time_us":1572,"prepare.run_cpu_time_us":624,"prepare.run_wall_time_us":1062,"replication_time_us":4614,"thread_start_us":8130,"threads_started":1}]]}
W20261118 14:58:12.240530 740 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.130:50135 (request call id 20) took 825 ms (client timeout 1000 ms). Trace:
W20261118 14:58:12.240630 740 rpcz_store.cc:269] 1118 14:58:11.414651 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:12.239347 (+824696us) service_pool.cc:224] Handling call
1118 14:58:12.240472 (+ 1125us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20261118 14:58:12.240870 1569 tablet_bootstrap.cc:492] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
W20261118 14:58:12.241931 740 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.129:47561 (request call id 17) took 819 ms (client timeout 1000 ms). Trace:
W20261118 14:58:12.242028 740 rpcz_store.cc:269] 1118 14:58:11.422684 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:12.240815 (+818131us) service_pool.cc:224] Handling call
1118 14:58:12.241876 (+ 1061us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20261118 14:58:12.242702 1569 tablet_bootstrap.cc:654] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:12.251873 740 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_ced554c65cd4465d9ddab052dc32efcf" } modify_external_catalogs: true
I20261118 14:58:12.252240 740 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39006:
table { table_name: "default.Table_ced554c65cd4465d9ddab052dc32efcf" } modify_external_catalogs: true
I20261118 14:58:12.260334 1573 tablet_bootstrap.cc:492] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:12.262288 1573 tablet_bootstrap.cc:654] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:12.271924 1570 tablet_bootstrap.cc:492] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:12.273772 1570 tablet_bootstrap.cc:654] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:12.273844 1573 tablet_bootstrap.cc:492] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:12.274005 1573 ts_tablet_manager.cc:1449] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.003s
I20261118 14:58:12.274729 1569 tablet_bootstrap.cc:492] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:12.274974 1569 ts_tablet_manager.cc:1449] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.019s
I20261118 14:58:12.274883 1573 raft_consensus.cc:360] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.275066 1573 raft_consensus.cc:386] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:12.275158 1573 raft_consensus.cc:748] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:12.275398 1573 consensus_queue.cc:261] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.276131 1573 ts_tablet_manager.cc:1480] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:12.277122 1569 raft_consensus.cc:360] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.277369 1569 raft_consensus.cc:386] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:12.277498 1569 raft_consensus.cc:748] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:12.277844 1569 consensus_queue.cc:261] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.281590 1569 ts_tablet_manager.cc:1480] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20261118 14:58:12.282639 1570 tablet_bootstrap.cc:492] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:12.282861 1570 ts_tablet_manager.cc:1449] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20261118 14:58:12.283720 1570 raft_consensus.cc:360] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.283973 1570 raft_consensus.cc:386] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:12.284083 1570 raft_consensus.cc:748] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:12.284317 1570 consensus_queue.cc:261] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.284947 1570 ts_tablet_manager.cc:1480] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261118 14:58:12.303813 741 catalog_manager.cc:2215] created HMS catalog entry for table [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:12.310766 741 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:39006 (request call id 20) took 2842 ms. Trace:
I20261118 14:58:12.310890 741 rpcz_store.cc:276] 1118 14:58:09.467776 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:09.683815 (+216039us) service_pool.cc:224] Handling call
1118 14:58:11.087666 (+1403851us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:11.088092 (+ 426us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:12.303759 (+1215667us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:12.310457 (+ 6698us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 14:58:12.310585 (+ 128us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 14:58:12.310695 (+ 110us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:12.304630 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 14:58:12.304768 (+ 138us) write_op.cc:432] Acquiring schema lock in shared mode
1118 14:58:12.304775 (+ 7us) write_op.cc:435] Acquired schema lock
1118 14:58:12.304777 (+ 2us) tablet.cc:670] Decoding operations
1118 14:58:12.304880 (+ 103us) write_op.cc:620] Acquiring the partition lock for write op
1118 14:58:12.304894 (+ 14us) write_op.cc:641] Partition lock acquired for write op
1118 14:58:12.304900 (+ 6us) tablet.cc:693] Acquiring locks for 2 operations
1118 14:58:12.304985 (+ 85us) tablet.cc:709] Row locks acquired
1118 14:58:12.304988 (+ 3us) write_op.cc:260] PREPARE: finished
1118 14:58:12.305029 (+ 41us) write_op.cc:270] Start()
1118 14:58:12.305064 (+ 35us) write_op.cc:276] Timestamp: P: 1795013892305003 usec, L: 0
1118 14:58:12.305068 (+ 4us) op_driver.cc:350] REPLICATION: starting
1118 14:58:12.305270 (+ 202us) log.cc:844] Serialized 432 byte log entry
1118 14:58:12.309381 (+ 4111us) op_driver.cc:466] REPLICATION: finished
1118 14:58:12.309481 (+ 100us) write_op.cc:301] APPLY: starting
1118 14:58:12.309498 (+ 17us) tablet.cc:1402] starting BulkCheckPresence
1118 14:58:12.309554 (+ 56us) tablet.cc:1405] finished BulkCheckPresence
1118 14:58:12.309580 (+ 26us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 14:58:12.309683 (+ 103us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 14:58:12.309718 (+ 35us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 14:58:12.309725 (+ 7us) write_op.cc:312] APPLY: finished
1118 14:58:12.309798 (+ 73us) log.cc:844] Serialized 28 byte log entry
1118 14:58:12.309859 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:12.309880 (+ 21us) write_op.cc:454] Released schema lock
1118 14:58:12.309887 (+ 7us) write_op.cc:341] FINISH: Updating metrics
1118 14:58:12.309953 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:12.309957 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1117422,"Hive Metastore.run_cpu_time_us":358,"Hive Metastore.run_wall_time_us":98037,"child_traces":[["op",{"apply.queue_time_us":55,"apply.run_cpu_time_us":464,"apply.run_wall_time_us":463,"num_ops":2,"prepare.queue_time_us":72,"prepare.run_cpu_time_us":779,"prepare.run_wall_time_us":779,"raft.queue_time_us":1236,"raft.run_cpu_time_us":526,"raft.run_wall_time_us":524,"replication_time_us":4286}]]}
I20261118 14:58:12.322607 1192 tablet_service.cc:1500] Processing CreateTablet for tablet 309f067161fe469dbb29d14c16139058 (DEFAULT_TABLE table=default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:12.322729 1057 tablet_service.cc:1500] Processing CreateTablet for tablet 309f067161fe469dbb29d14c16139058 (DEFAULT_TABLE table=default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:12.323072 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309f067161fe469dbb29d14c16139058. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:12.323130 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309f067161fe469dbb29d14c16139058. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:12.325265 741 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d] alter complete (version 1)
I20261118 14:58:12.328488 741 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39006:
name: "default.Table_4d2c8b2bcb994782a8bbf4b38ca2f7af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:12.328919 918 tablet_service.cc:1500] Processing CreateTablet for tablet 309f067161fe469dbb29d14c16139058 (DEFAULT_TABLE table=default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:12.329319 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 309f067161fe469dbb29d14c16139058. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:12.333146 1569 tablet_bootstrap.cc:492] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:12.334900 1569 tablet_bootstrap.cc:654] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:12.338672 1570 tablet_bootstrap.cc:492] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:12.339708 1573 tablet_bootstrap.cc:492] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:12.340457 1570 tablet_bootstrap.cc:654] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:12.342000 1573 tablet_bootstrap.cc:654] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:12.358894 1570 tablet_bootstrap.cc:492] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:12.359205 1570 ts_tablet_manager.cc:1449] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.002s
I20261118 14:58:12.360107 1570 raft_consensus.cc:360] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.360388 1570 raft_consensus.cc:386] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:12.360503 1570 raft_consensus.cc:748] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:12.360852 1570 consensus_queue.cc:261] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.363080 1570 ts_tablet_manager.cc:1480] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20261118 14:58:12.369892 1573 tablet_bootstrap.cc:492] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:12.370085 1573 ts_tablet_manager.cc:1449] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.030s user 0.013s sys 0.009s
I20261118 14:58:12.370893 1573 raft_consensus.cc:360] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.371096 1573 raft_consensus.cc:386] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:12.371170 1573 raft_consensus.cc:748] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:12.373044 1573 consensus_queue.cc:261] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.378546 1573 ts_tablet_manager.cc:1480] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20261118 14:58:12.380599 1560 raft_consensus.cc:499] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:12.380757 1560 raft_consensus.cc:521] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.381419 1560 leader_election.cc:290] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:12.382004 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "309f067161fe469dbb29d14c16139058" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:12.382113 1569 tablet_bootstrap.cc:492] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:12.382112 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "309f067161fe469dbb29d14c16139058" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:12.383132 1569 ts_tablet_manager.cc:1449] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.049s user 0.017s sys 0.009s
I20261118 14:58:12.383191 1210 raft_consensus.cc:2490] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
W20261118 14:58:12.383373 875 leader_election.cc:343] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Illegal state: must be running to vote when last-logged opid is not known
I20261118 14:58:12.383687 874 leader_election.cc:304] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d
I20261118 14:58:12.383942 1560 raft_consensus.cc:2826] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:12.384063 1560 raft_consensus.cc:499] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:12.384160 1560 raft_consensus.cc:3096] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:12.384565 1569 raft_consensus.cc:360] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.384747 1569 raft_consensus.cc:386] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:12.384816 1569 raft_consensus.cc:748] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:12.385283 1569 consensus_queue.cc:261] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.385785 1569 ts_tablet_manager.cc:1480] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20261118 14:58:12.397703 1560 raft_consensus.cc:521] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.398522 1560 leader_election.cc:290] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:12.399359 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "309f067161fe469dbb29d14c16139058" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:12.399570 1076 raft_consensus.cc:3096] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:12.399772 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "309f067161fe469dbb29d14c16139058" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:12.399989 1210 raft_consensus.cc:3096] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:12.402484 1076 raft_consensus.cc:2490] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:12.402976 875 leader_election.cc:304] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:12.403549 1560 raft_consensus.cc:2826] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:12.403712 1560 raft_consensus.cc:705] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:12.404018 1560 consensus_queue.cc:238] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.414103 1331 raft_consensus.cc:499] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:12.414314 1331 raft_consensus.cc:521] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.415175 1331 leader_election.cc:290] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:12.415970 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "dd0cddf6622a469eb598e9740407a9f0" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:12.416193 937 raft_consensus.cc:2490] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:12.416780 1010 leader_election.cc:304] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:12.417155 1331 raft_consensus.cc:2826] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:12.417269 1331 raft_consensus.cc:499] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:12.417439 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "dd0cddf6622a469eb598e9740407a9f0" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:12.417716 1211 raft_consensus.cc:2490] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:12.417901 1331 raft_consensus.cc:3096] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:12.421334 1560 consensus_queue.cc:1049] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:12.489271 1210 raft_consensus.cc:2490] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:12.489274 1331 raft_consensus.cc:521] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.490003 1331 leader_election.cc:290] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:12.490516 1581 consensus_queue.cc:1049] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:12.491017 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "dd0cddf6622a469eb598e9740407a9f0" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:12.491145 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "dd0cddf6622a469eb598e9740407a9f0" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:12.491204 937 raft_consensus.cc:3096] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:12.491322 1211 raft_consensus.cc:3096] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:12.581884 1211 raft_consensus.cc:2490] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:12.581885 937 raft_consensus.cc:2490] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:12.582540 1012 leader_election.cc:304] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:12.582901 1331 raft_consensus.cc:2826] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:12.583074 1331 raft_consensus.cc:705] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:12.583516 1331 consensus_queue.cc:238] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:12.587625 736 catalog_manager.cc:3543] renamed table default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0] in HMS: new name default.table_3f41e3f42a794945838124ce6712ceb3
I20261118 14:58:12.588650 1583 consensus_queue.cc:1049] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:12.589893 1590 consensus_queue.cc:1049] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:12.822109 1594 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:12.822444 1594 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:12.824551 1594 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:12.825178 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 2 candidate_status { last_received { term: 1 index: 54 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:12.825676 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 2 candidate_status { last_received { term: 1 index: 54 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e" is_pre_election: true
I20261118 14:58:12.826294 1463 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c9ef96ff074450a879e66787486386d; no voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:12.827184 1594 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:12.838299 1595 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:12.838548 1595 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:12.839308 1595 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:12.839785 1489 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 54 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d" is_pre_election: true
I20261118 14:58:12.839824 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 54 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:12.840062 1489 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed65a3719634e519d783540afd6189e in term 1.
I20261118 14:58:12.840552 588 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d; no voters: 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:12.840821 1595 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261118 14:58:12.840925 1595 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:12.841014 1595 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 1 FOLLOWER]: Advancing to term 2
I20261118 14:58:12.902976 1595 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:12.903478 613 raft_consensus.cc:1250] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 2 FOLLOWER]: Rejecting Update request from peer 9b90720bc0b34e4994abf4fc6f9e59e1 for earlier term 1. Current term is 2. Ops: []
I20261118 14:58:12.903818 1595 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 election: Requested vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:12.904297 1489 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 54 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d"
I20261118 14:58:12.904413 1596 consensus_queue.cc:1060] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 }, Status: INVALID_TERM, Last received: 1.54, Next index: 55, Last known committed idx: 54, Time since last communication: 0.000s
I20261118 14:58:12.904542 1489 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 1 FOLLOWER]: Advancing to term 2
I20261118 14:58:12.904613 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 2 candidate_status { last_received { term: 1 index: 54 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
I20261118 14:58:12.905056 1597 raft_consensus.cc:3091] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 1 LEADER]: Stepping down as leader of term 1
I20261118 14:58:12.905157 1597 raft_consensus.cc:748] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9b90720bc0b34e4994abf4fc6f9e59e1, State: Running, Role: LEADER
I20261118 14:58:12.905498 1597 consensus_queue.cc:261] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 54, Committed index: 54, Last appended: 1.54, Last appended by leader: 54, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:12.905913 1597 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 1 FOLLOWER]: Advancing to term 2
I20261118 14:58:12.996758 1489 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed65a3719634e519d783540afd6189e in term 2.
I20261118 14:58:12.997373 588 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d; no voters: 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:12.997776 1595 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 2 FOLLOWER]: Leader election won for term 2
I20261118 14:58:12.998147 1595 raft_consensus.cc:705] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 2 LEADER]: Becoming Leader. State: Replica: 0ed65a3719634e519d783540afd6189e, State: Running, Role: LEADER
I20261118 14:58:12.998629 1595 consensus_queue.cc:238] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 54, Committed index: 54, Last appended: 1.54, Last appended by leader: 54, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:13.000039 1598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ed65a3719634e519d783540afd6189e. Latest consensus state: current_term: 2 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:13.000473 1598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:13.001314 1601 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 14:58:13.001864 1489 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Refusing update from remote peer 0ed65a3719634e519d783540afd6189e: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 2 index: 55. (index mismatch)
I20261118 14:58:13.002046 753 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 2 FOLLOWER]: Refusing update from remote peer 0ed65a3719634e519d783540afd6189e: Log matching property violated. Preceding OpId in replica: term: 1 index: 54. Preceding OpId from leader: term: 2 index: 55. (index mismatch)
I20261118 14:58:13.002590 1600 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 54, Time since last communication: 0.000s
I20261118 14:58:13.002853 1599 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 54, Time since last communication: 0.000s
I20261118 14:58:13.004243 1594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ed65a3719634e519d783540afd6189e. Latest consensus state: current_term: 2 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:13.004444 1594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:13.004495 1597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ed65a3719634e519d783540afd6189e. Latest consensus state: current_term: 2 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:13.004628 1597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:13.006048 1601 catalog_manager.cc:694] Loaded metadata for table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:13.006485 1601 catalog_manager.cc:694] Loaded metadata for table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]
I20261118 14:58:13.006826 1601 catalog_manager.cc:694] Loaded metadata for table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:13.007162 1601 catalog_manager.cc:694] Loaded metadata for table default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:13.007467 1601 catalog_manager.cc:694] Loaded metadata for table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]
I20261118 14:58:13.007795 1601 catalog_manager.cc:694] Loaded metadata for table default.table_59239feaba874d6890fe8e86d57545e2 [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:13.008093 1601 catalog_manager.cc:694] Loaded metadata for table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:13.008404 1601 catalog_manager.cc:694] Loaded metadata for table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]
I20261118 14:58:13.008713 1601 catalog_manager.cc:694] Loaded metadata for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:13.009017 1601 catalog_manager.cc:694] Loaded metadata for table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]
I20261118 14:58:13.009357 1601 catalog_manager.cc:694] Loaded metadata for table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]
I20261118 14:58:13.009747 1601 catalog_manager.cc:694] Loaded metadata for table default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7]
I20261118 14:58:13.011901 1601 tablet_loader.cc:96] loaded metadata for tablet 309f067161fe469dbb29d14c16139058 (table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15])
I20261118 14:58:13.012216 1601 tablet_loader.cc:96] loaded metadata for tablet 75581812cbcb48e5a2ea9d0fb925c03a (table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe])
I20261118 14:58:13.012486 1601 tablet_loader.cc:96] loaded metadata for tablet 8a07aac51e8a48dba2c61e9e477fad11 (table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f])
I20261118 14:58:13.012929 1601 tablet_loader.cc:96] loaded metadata for tablet 91a4a7f508b44f809c8071413df8d648 (table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985])
I20261118 14:58:13.013253 1601 tablet_loader.cc:96] loaded metadata for tablet 92fab0337a8e4ffeb604589ac588b0af (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7])
I20261118 14:58:13.013916 1601 tablet_loader.cc:96] loaded metadata for tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d])
I20261118 14:58:13.014184 1601 tablet_loader.cc:96] loaded metadata for tablet cdc9a8de0308404fb2fa23b70984f12b (table default.table_38deaf819a5e430090d96b02cbd9cdc3 [id=51a7f645ca6b4e56affbfecd1f1187a0])
I20261118 14:58:13.014488 1601 tablet_loader.cc:96] loaded metadata for tablet cf7800d54792490faa3860de0ecb7a3a (table default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7])
I20261118 14:58:13.014864 1601 tablet_loader.cc:96] loaded metadata for tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6])
I20261118 14:58:13.015116 1601 tablet_loader.cc:96] loaded metadata for tablet e47d7d10a5c741ecb30ec701c76b27a3 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f])
I20261118 14:58:13.015744 1601 tablet_loader.cc:96] loaded metadata for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b])
I20261118 14:58:13.016007 1601 tablet_loader.cc:96] loaded metadata for tablet f66c4e0f06f043108a6451666a80a5f9 (table default.table_59239feaba874d6890fe8e86d57545e2 [id=886aa9bb52444e1186957fd7e610fb28])
I20261118 14:58:13.016202 1601 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:13.017062 1601 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:13.017125 1601 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:13.018349 1601 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:13.019237 1601 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Loaded TSK: 0
I20261118 14:58:13.020038 1601 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:13.020877 1601 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:13.021759 1601 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 12
I20261118 14:58:13.028205 1598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:13.028396 1598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:13.027928 1599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:13.028818 1599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:13.029649 1594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:13.029846 1594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:13.030001 1597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:13.030208 1597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:13.111433 601 catalog_manager.cc:5840] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129). New cstate: current_term: 1 leader_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: UNKNOWN } } }
I20261118 14:58:13.125631 985 heartbeater.cc:500] Master 127.0.72.190:42447 was elected leader, sending a full tablet report...
I20261118 14:58:13.128916 601 catalog_manager.cc:5840] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c4fe5e44448d98db1eed6329d150d (127.0.72.130). New cstate: current_term: 1 leader_uuid: "245c4fe5e44448d98db1eed6329d150d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } }
I20261118 14:58:13.324808 803 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:13.335908 1123 heartbeater.cc:500] Master 127.0.72.190:42447 was elected leader, sending a full tablet report...
I20261118 14:58:13.426707 735 catalog_manager.cc:2215] created HMS catalog entry for table [id=ac2eba7d96da4b819c14599278173309]
W20261118 14:58:13.427645 735 catalog_manager.cc:2243] Illegal state: an error occurred while writing to the sys-catalog: replica 9b90720bc0b34e4994abf4fc6f9e59e1 is not leader of this config: current role FOLLOWER
I20261118 14:58:13.609089 1258 heartbeater.cc:500] Master 127.0.72.190:42447 was elected leader, sending a full tablet report...
I20261118 14:58:13.619796 738 catalog_manager.cc:3543] renamed table default.table_7886615812a14a0492f39f2694999785 [id=fae4a508ca0b4035a4843cdcfa56f4e7] in HMS: new name default.table_927d6a72d3524190964efb6e9f1f8d42
I20261118 14:58:13.634763 1611 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:13.634987 1611 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:13.635841 1611 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:13.636337 753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 3 candidate_status { last_received { term: 2 index: 57 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:13.636833 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 3 candidate_status { last_received { term: 2 index: 57 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e" is_pre_election: true
I20261118 14:58:13.637341 1463 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c9ef96ff074450a879e66787486386d; no voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:13.637774 1611 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20261118 14:58:13.651474 290 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu pid 709
I20261118 14:58:13.652547 773 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 14:58:13.785727 737 catalog_manager.cc:2215] created HMS catalog entry for table [id=9d4754075da243c49b8824356549bb98]
I20261118 14:58:13.786521 290 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu with pid 709
I20261118 14:58:13.814096 572 master_runner.cc:187] Error getting master registration for 127.0.72.188:33967: Network error: recv got EOF from 127.0.72.188:33967 (error 108),
I20261118 14:58:13.814302 572 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:58:13.815315 1266 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.188:33967)
W20261118 14:58:13.815788 1266 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.188:33967)
W20261118 14:58:13.816078 1266 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.188:33967)
W20261118 14:58:13.816834 1266 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.188:33967)
I20261118 14:58:13.813900 1355 master_runner.cc:187] Error getting master registration for 127.0.72.188:33967: Network error: recv got EOF from 127.0.72.188:33967 (error 108),
I20261118 14:58:13.818849 1355 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:13.822329 601 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60882:
name: "default.Table_a0ea13e08f4e45bc84d5936c615c8ff7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:13.823472 594 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60882:
name: "default.Table_4dba69207cca45529e828162200c13f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:13.823989 595 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60882:
name: "default.Table_d24daba84f8c4c94824da319564f0453"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:13.824477 596 master_service.cc:965] ReplaceTablet: received request to replace tablet cdc9a8de0308404fb2fa23b70984f12b from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:13.825062 597 master_service.cc:965] ReplaceTablet: received request to replace tablet 92fab0337a8e4ffeb604589ac588b0af from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:13.826522 599 master_service.cc:965] ReplaceTablet: received request to replace tablet f4bdcca79fcb4fedaea1fdbe2dd31112 from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:13.826844 602 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60882:
name: "default.Table_36b9017c9ae24ecdb8ae81bf85341052"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:13.827541 600 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60882:
table { table_name: "default.Table_537f6e87a8d040a695fda278ff06cbf5" } new_table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" modify_external_catalogs: true
I20261118 14:58:13.827957 598 master_service.cc:965] ReplaceTablet: received request to replace tablet 91a4a7f508b44f809c8071413df8d648 from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:13.826581 603 master_service.cc:965] ReplaceTablet: received request to replace tablet 07d0bc6b6bb6484892091f81ecdc505a from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:13.829919 603 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60882:
table { table_name: "default.Table_6c4a501fa569491080a3b4988350b5b3" } modify_external_catalogs: true
I20261118 14:58:13.830117 603 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60882:
table { table_name: "default.Table_6c4a501fa569491080a3b4988350b5b3" } modify_external_catalogs: true
W20261118 14:58:13.841013 589 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20261118 14:58:13.842928 589 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e -> Peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Couldn't send request to peer 9b90720bc0b34e4994abf4fc6f9e59e1. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:13.845273 596 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Sending DeleteTablet for 3 replicas of tablet cdc9a8de0308404fb2fa23b70984f12b
I20261118 14:58:13.846102 597 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Sending DeleteTablet for 3 replicas of tablet 92fab0337a8e4ffeb604589ac588b0af
I20261118 14:58:13.846632 599 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Sending DeleteTablet for 3 replicas of tablet f4bdcca79fcb4fedaea1fdbe2dd31112
I20261118 14:58:13.848069 599 catalog_manager.cc:6936] ReplaceTablet: tablet f4bdcca79fcb4fedaea1fdbe2dd31112 deleted and replaced by tablet 86c2240a7e3d415e92776000511c7557
I20261118 14:58:13.848546 597 catalog_manager.cc:6936] ReplaceTablet: tablet 92fab0337a8e4ffeb604589ac588b0af deleted and replaced by tablet 212e4a39f2d0465a854b3474dc020aa6
I20261118 14:58:13.849291 596 catalog_manager.cc:6936] ReplaceTablet: tablet cdc9a8de0308404fb2fa23b70984f12b deleted and replaced by tablet a00587922d884ced8250ac130789da68
I20261118 14:58:13.878432 918 tablet_service.cc:1547] Processing DeleteTablet for tablet cdc9a8de0308404fb2fa23b70984f12b with delete_type TABLET_DATA_DELETED (replaced by tablet a00587922d884ced8250ac130789da68) from {username='slave'} at 127.0.0.1:54852
I20261118 14:58:13.879091 1633 tablet_replica.cc:333] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:13.879354 918 tablet_service.cc:1547] Processing DeleteTablet for tablet 92fab0337a8e4ffeb604589ac588b0af with delete_type TABLET_DATA_DELETED (replaced by tablet 212e4a39f2d0465a854b3474dc020aa6) from {username='slave'} at 127.0.0.1:54852
I20261118 14:58:13.879508 1633 raft_consensus.cc:2261] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:13.879828 918 tablet_service.cc:1547] Processing DeleteTablet for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 with delete_type TABLET_DATA_DELETED (replaced by tablet 86c2240a7e3d415e92776000511c7557) from {username='slave'} at 127.0.0.1:54852
I20261118 14:58:13.880775 1633 raft_consensus.cc:2290] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:13.881492 1633 ts_tablet_manager.cc:1962] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:13.883134 597 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60882:
name: "default.Table_4789eba8c66842f6ae326364062589fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:13.884119 598 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Sending DeleteTablet for 3 replicas of tablet 91a4a7f508b44f809c8071413df8d648
I20261118 14:58:13.884701 598 catalog_manager.cc:6936] ReplaceTablet: tablet 91a4a7f508b44f809c8071413df8d648 deleted and replaced by tablet 1b82ed8e1b7244e7b28fed1b93bec69c
I20261118 14:58:13.886304 918 tablet_service.cc:1547] Processing DeleteTablet for tablet 91a4a7f508b44f809c8071413df8d648 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b82ed8e1b7244e7b28fed1b93bec69c) from {username='slave'} at 127.0.0.1:54852
I20261118 14:58:13.886659 599 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60882:
table { table_name: "default.Table_41739c72be484e8fb242c925dd0e2c32" } modify_external_catalogs: true
I20261118 14:58:13.886736 1192 tablet_service.cc:1547] Processing DeleteTablet for tablet 92fab0337a8e4ffeb604589ac588b0af with delete_type TABLET_DATA_DELETED (replaced by tablet 212e4a39f2d0465a854b3474dc020aa6) from {username='slave'} at 127.0.0.1:60868
I20261118 14:58:13.886871 599 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60882:
table { table_name: "default.Table_41739c72be484e8fb242c925dd0e2c32" } modify_external_catalogs: true
I20261118 14:58:13.887193 1189 tablet_service.cc:1547] Processing DeleteTablet for tablet 91a4a7f508b44f809c8071413df8d648 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b82ed8e1b7244e7b28fed1b93bec69c) from {username='slave'} at 127.0.0.1:60868
I20261118 14:58:13.887775 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet cdc9a8de0308404fb2fa23b70984f12b with delete_type TABLET_DATA_DELETED (replaced by tablet a00587922d884ced8250ac130789da68) from {username='slave'} at 127.0.0.1:60868
I20261118 14:58:13.888017 1191 tablet_service.cc:1547] Processing DeleteTablet for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 with delete_type TABLET_DATA_DELETED (replaced by tablet 86c2240a7e3d415e92776000511c7557) from {username='slave'} at 127.0.0.1:60868
I20261118 14:58:13.888478 596 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60882:
table { table_name: "default.Table_d908b160aac24b678c5aff465735ea9f" } modify_external_catalogs: true
I20261118 14:58:13.888742 596 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60882:
table { table_name: "default.Table_d908b160aac24b678c5aff465735ea9f" } modify_external_catalogs: true
I20261118 14:58:13.890806 598 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60882:
name: "default.Table_053874e1edfe4b8eac52b4ba5f1b1870"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:13.896227 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 92fab0337a8e4ffeb604589ac588b0af with delete_type TABLET_DATA_DELETED (replaced by tablet 212e4a39f2d0465a854b3474dc020aa6) from {username='slave'} at 127.0.0.1:49486
I20261118 14:58:13.896364 1054 tablet_service.cc:1547] Processing DeleteTablet for tablet 91a4a7f508b44f809c8071413df8d648 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b82ed8e1b7244e7b28fed1b93bec69c) from {username='slave'} at 127.0.0.1:49486
I20261118 14:58:13.896659 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet f4bdcca79fcb4fedaea1fdbe2dd31112 with delete_type TABLET_DATA_DELETED (replaced by tablet 86c2240a7e3d415e92776000511c7557) from {username='slave'} at 127.0.0.1:49486
I20261118 14:58:13.896938 1055 tablet_service.cc:1547] Processing DeleteTablet for tablet cdc9a8de0308404fb2fa23b70984f12b with delete_type TABLET_DATA_DELETED (replaced by tablet a00587922d884ced8250ac130789da68) from {username='slave'} at 127.0.0.1:49486
I20261118 14:58:13.898814 1637 tablet_replica.cc:333] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:13.899041 1634 tablet_replica.cc:333] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:13.899358 1634 raft_consensus.cc:2261] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:13.899669 1634 raft_consensus.cc:2290] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20261118 14:58:13.900192 826 catalog_manager.cc:6108] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a00587922d884ced8250ac130789da68 has no consensus state
I20261118 14:58:13.900178 1637 raft_consensus.cc:2261] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:13.900460 1634 ts_tablet_manager.cc:1962] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:13.900617 1637 raft_consensus.cc:2290] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:13.901194 1637 ts_tablet_manager.cc:1962] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:13.906925 1057 tablet_service.cc:1500] Processing CreateTablet for tablet 86c2240a7e3d415e92776000511c7557 (DEFAULT_TABLE table=default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.907258 918 tablet_service.cc:1500] Processing CreateTablet for tablet 86c2240a7e3d415e92776000511c7557 (DEFAULT_TABLE table=default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.907821 919 tablet_service.cc:1500] Processing CreateTablet for tablet 1b82ed8e1b7244e7b28fed1b93bec69c (DEFAULT_TABLE table=default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.908143 917 tablet_service.cc:1500] Processing CreateTablet for tablet a00587922d884ced8250ac130789da68 (DEFAULT_TABLE table=default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.909551 916 tablet_service.cc:1500] Processing CreateTablet for tablet 212e4a39f2d0465a854b3474dc020aa6 (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.910104 1054 tablet_service.cc:1500] Processing CreateTablet for tablet 212e4a39f2d0465a854b3474dc020aa6 (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.910179 1056 tablet_service.cc:1500] Processing CreateTablet for tablet a00587922d884ced8250ac130789da68 (DEFAULT_TABLE table=default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.910655 1055 tablet_service.cc:1500] Processing CreateTablet for tablet 1b82ed8e1b7244e7b28fed1b93bec69c (DEFAULT_TABLE table=default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.911289 1192 tablet_service.cc:1500] Processing CreateTablet for tablet 1b82ed8e1b7244e7b28fed1b93bec69c (DEFAULT_TABLE table=default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.911671 1189 tablet_service.cc:1500] Processing CreateTablet for tablet 212e4a39f2d0465a854b3474dc020aa6 (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.911811 1190 tablet_service.cc:1500] Processing CreateTablet for tablet a00587922d884ced8250ac130789da68 (DEFAULT_TABLE table=default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:13.914759 1191 tablet_service.cc:1500] Processing CreateTablet for tablet 86c2240a7e3d415e92776000511c7557 (DEFAULT_TABLE table=default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
W20261118 14:58:13.918792 601 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:13.919291 1647 tablet.cc:1759] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:13.919885 1645 tablet.cc:1759] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:13.929265 1651 tablet.cc:1759] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:13.930871 917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a00587922d884ced8250ac130789da68. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.931613 916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 212e4a39f2d0465a854b3474dc020aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.931828 1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 212e4a39f2d0465a854b3474dc020aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.932075 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b82ed8e1b7244e7b28fed1b93bec69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.932382 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a00587922d884ced8250ac130789da68. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.932487 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c2240a7e3d415e92776000511c7557. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.932848 1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b82ed8e1b7244e7b28fed1b93bec69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.934022 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c2240a7e3d415e92776000511c7557. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.938547 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b82ed8e1b7244e7b28fed1b93bec69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.939043 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 212e4a39f2d0465a854b3474dc020aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.939496 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a00587922d884ced8250ac130789da68. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:13.939904 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c2240a7e3d415e92776000511c7557. 1 dirs total, 0 dirs full, 0 dirs failed
W20261118 14:58:13.943185 587 catalog_manager.cc:4847] Async tablet task AlterTable RPC for tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a00587922d884ced8250ac130789da68 consensus state has no leader
I20261118 14:58:13.956137 1655 tablet_bootstrap.cc:492] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:13.958669 1652 tablet.cc:1759] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:13.960011 1653 tablet_bootstrap.cc:492] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:13.961320 1655 tablet_bootstrap.cc:654] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:13.963635 1653 tablet_bootstrap.cc:654] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:13.968307 1654 tablet.cc:1759] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:13.971225 1658 tablet.cc:1759] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:13.971935 1655 tablet_bootstrap.cc:492] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:13.971961 1653 tablet_bootstrap.cc:492] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:13.972110 1653 ts_tablet_manager.cc:1449] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20261118 14:58:13.972121 1655 ts_tablet_manager.cc:1449] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.002s
I20261118 14:58:13.972748 1653 raft_consensus.cc:360] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.972776 1655 raft_consensus.cc:360] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.972901 1653 raft_consensus.cc:386] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:13.972944 1655 raft_consensus.cc:386] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:13.972968 1653 raft_consensus.cc:748] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:13.973053 1655 raft_consensus.cc:748] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:13.973192 1653 consensus_queue.cc:261] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.973714 1653 ts_tablet_manager.cc:1480] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20261118 14:58:13.974210 1653 tablet_bootstrap.cc:492] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:13.973289 1655 consensus_queue.cc:261] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.976485 1655 ts_tablet_manager.cc:1480] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20261118 14:58:13.976814 1653 tablet_bootstrap.cc:654] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:13.978081 1655 tablet_bootstrap.cc:492] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:13.980141 1655 tablet_bootstrap.cc:654] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:13.981823 1653 tablet_bootstrap.cc:492] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:13.981977 1653 ts_tablet_manager.cc:1449] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20261118 14:58:13.982327 1655 tablet_bootstrap.cc:492] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:13.982520 1655 ts_tablet_manager.cc:1449] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20261118 14:58:13.982605 1653 raft_consensus.cc:360] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.982746 1653 raft_consensus.cc:386] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:13.982796 1653 raft_consensus.cc:748] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:13.982985 1653 consensus_queue.cc:261] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.983381 1653 ts_tablet_manager.cc:1480] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20261118 14:58:13.983484 1655 raft_consensus.cc:360] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.983690 1655 raft_consensus.cc:386] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:13.983722 1653 tablet_bootstrap.cc:492] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:13.983762 1655 raft_consensus.cc:748] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:13.984040 1655 consensus_queue.cc:261] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.984776 1655 ts_tablet_manager.cc:1480] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 14:58:13.985301 1653 tablet_bootstrap.cc:654] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:13.985397 1655 tablet_bootstrap.cc:492] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:13.987135 1653 tablet_bootstrap.cc:492] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:13.987255 1653 ts_tablet_manager.cc:1449] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20261118 14:58:13.987474 1655 tablet_bootstrap.cc:654] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:13.987847 1653 raft_consensus.cc:360] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.988639 1653 raft_consensus.cc:386] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:13.988742 1653 raft_consensus.cc:748] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:13.988963 1653 consensus_queue.cc:261] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.989399 1653 ts_tablet_manager.cc:1480] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261118 14:58:13.989851 1653 tablet_bootstrap.cc:492] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:13.991278 1653 tablet_bootstrap.cc:654] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:13.995513 1655 tablet_bootstrap.cc:492] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:13.995561 1653 tablet_bootstrap.cc:492] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:13.995653 1655 ts_tablet_manager.cc:1449] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20261118 14:58:13.995677 1653 ts_tablet_manager.cc:1449] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20261118 14:58:13.996169 1653 raft_consensus.cc:360] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.996300 1653 raft_consensus.cc:386] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:13.996364 1653 raft_consensus.cc:748] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:13.996371 1655 raft_consensus.cc:360] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.996557 1655 raft_consensus.cc:386] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:13.996666 1655 raft_consensus.cc:748] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:13.996588 1653 consensus_queue.cc:261] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.996910 1655 consensus_queue.cc:261] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:13.997108 1653 ts_tablet_manager.cc:1480] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20261118 14:58:13.998332 1655 ts_tablet_manager.cc:1480] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20261118 14:58:13.998711 1655 tablet_bootstrap.cc:492] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:14.000763 1655 tablet_bootstrap.cc:654] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:14.002856 1655 tablet_bootstrap.cc:492] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:14.003000 1655 ts_tablet_manager.cc:1449] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20261118 14:58:14.003688 1655 raft_consensus.cc:360] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.003836 1655 raft_consensus.cc:386] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:14.003894 1655 raft_consensus.cc:748] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:14.004123 1655 consensus_queue.cc:261] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.004611 1655 ts_tablet_manager.cc:1480] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20261118 14:58:14.024230 1608 raft_consensus.cc:499] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:14.024358 1608 raft_consensus.cc:521] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.025010 1608 leader_election.cc:290] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:14.025378 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "86c2240a7e3d415e92776000511c7557" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:14.025563 938 raft_consensus.cc:2490] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
W20261118 14:58:14.025821 1012 leader_election.cc:343] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Not found: Tablet not found: 86c2240a7e3d415e92776000511c7557
I20261118 14:58:14.026007 1010 leader_election.cc:304] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters: c01f1b88968f4905ba5a401a81ade9f7
I20261118 14:58:14.026264 1608 raft_consensus.cc:2826] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:14.026381 1608 raft_consensus.cc:499] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:14.026464 1608 raft_consensus.cc:3096] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.030014 1637 ts_tablet_manager.cc:1975] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.030010 1634 ts_tablet_manager.cc:1975] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.030014 1633 ts_tablet_manager.cc:1975] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.030122 1637 log.cc:1199] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/92fab0337a8e4ffeb604589ac588b0af
I20261118 14:58:14.030130 1634 log.cc:1199] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/92fab0337a8e4ffeb604589ac588b0af
I20261118 14:58:14.030134 1633 log.cc:1199] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/cdc9a8de0308404fb2fa23b70984f12b
I20261118 14:58:14.030537 1637 ts_tablet_manager.cc:1996] T 92fab0337a8e4ffeb604589ac588b0af P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:14.030575 1633 ts_tablet_manager.cc:1996] T cdc9a8de0308404fb2fa23b70984f12b P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:14.030596 1634 ts_tablet_manager.cc:1996] T 92fab0337a8e4ffeb604589ac588b0af P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:14.031472 1634 tablet_replica.cc:333] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:14.031471 1633 tablet_replica.cc:333] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:14.031661 1634 raft_consensus.cc:2261] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:14.031661 1633 raft_consensus.cc:2261] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:14.031813 1633 raft_consensus.cc:2290] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:14.031813 1634 raft_consensus.cc:2290] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:14.032330 1633 ts_tablet_manager.cc:1962] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:14.032334 1634 ts_tablet_manager.cc:1962] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:14.034448 589 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 92fab0337a8e4ffeb604589ac588b0af (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]) successfully deleted
I20261118 14:58:14.036732 1608 raft_consensus.cc:521] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.037176 587 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet cdc9a8de0308404fb2fa23b70984f12b (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:14.037480 1608 leader_election.cc:290] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:14.037982 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "86c2240a7e3d415e92776000511c7557" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:14.038183 938 raft_consensus.cc:3096] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
W20261118 14:58:14.038581 1012 leader_election.cc:343] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Not found: Tablet not found: 86c2240a7e3d415e92776000511c7557
I20261118 14:58:14.041471 1665 tablet_bootstrap.cc:492] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:14.042930 1665 tablet_bootstrap.cc:654] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:14.044415 1665 tablet_bootstrap.cc:492] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:14.044545 1665 ts_tablet_manager.cc:1449] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:14.045099 1665 raft_consensus.cc:360] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.045774 1665 raft_consensus.cc:386] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:14.045866 1665 raft_consensus.cc:748] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:14.046061 1665 consensus_queue.cc:261] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.047441 1665 ts_tablet_manager.cc:1480] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:14.136444 601 catalog_manager.cc:2215] created HMS catalog entry for table [id=69ddcff335ba4d85bba72a380a850627]
I20261118 14:58:14.141000 938 raft_consensus.cc:2490] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
W20261118 14:58:14.142663 594 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4dba69207cca45529e828162200c13f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:14.143117 1637 tablet_replica.cc:333] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
W20261118 14:58:14.143271 598 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_053874e1edfe4b8eac52b4ba5f1b1870 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:14.143333 1637 raft_consensus.cc:2261] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:14.143519 1637 raft_consensus.cc:2290] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
W20261118 14:58:14.143788 595 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d24daba84f8c4c94824da319564f0453 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:14.144111 1637 ts_tablet_manager.cc:1962] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20261118 14:58:14.144081 597 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4789eba8c66842f6ae326364062589fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:14.142663 602 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_36b9017c9ae24ecdb8ae81bf85341052 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:14.145133 1010 leader_election.cc:304] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters: c01f1b88968f4905ba5a401a81ade9f7
I20261118 14:58:14.146399 1608 raft_consensus.cc:2826] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:14.146817 590 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 92fab0337a8e4ffeb604589ac588b0af (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]) successfully deleted
I20261118 14:58:14.150636 1608 raft_consensus.cc:705] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:14.150974 1608 consensus_queue.cc:238] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.157640 1665 tablet_bootstrap.cc:492] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:14.159361 1665 tablet_bootstrap.cc:654] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:14.160873 1211 raft_consensus.cc:3096] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
W20261118 14:58:14.166147 875 consensus_peers.cc:597] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:14.167447 601 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
W20261118 14:58:14.168016 601 catalog_manager.cc:6040] Failed to send AlterTable RPC for tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 86c2240a7e3d415e92776000511c7557 consensus state has no leader
I20261118 14:58:14.168219 917 tablet_service.cc:1500] Processing CreateTablet for tablet f87ee223b5a246c58e9b2eca5740d08c (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:14.169118 601 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:14.173023 1189 tablet_service.cc:1500] Processing CreateTablet for tablet f87ee223b5a246c58e9b2eca5740d08c (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:14.176709 1668 consensus_queue.cc:1049] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20261118 14:58:14.177125 1055 tablet_service.cc:1500] Processing CreateTablet for tablet f87ee223b5a246c58e9b2eca5740d08c (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:14.182387 1665 tablet_bootstrap.cc:492] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:14.182621 1665 ts_tablet_manager.cc:1449] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.008s
I20261118 14:58:14.183470 1665 raft_consensus.cc:360] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.183713 1665 raft_consensus.cc:386] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:14.183821 1665 raft_consensus.cc:748] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:14.184105 1665 consensus_queue.cc:261] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.185806 1665 ts_tablet_manager.cc:1480] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:14.187271 1665 tablet_bootstrap.cc:492] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:14.189095 1665 tablet_bootstrap.cc:654] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:14.181285 601 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
I20261118 14:58:14.191009 1665 tablet_bootstrap.cc:492] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:14.191133 1665 ts_tablet_manager.cc:1449] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20261118 14:58:14.191882 601 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:14.191843 1665 raft_consensus.cc:360] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.192018 1665 raft_consensus.cc:386] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:14.192170 1665 raft_consensus.cc:748] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:14.192205 601 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
I20261118 14:58:14.192487 1665 consensus_queue.cc:261] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.196144 1665 ts_tablet_manager.cc:1480] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20261118 14:58:14.199681 601 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28] alter complete (version 1)
I20261118 14:58:14.201812 601 catalog_manager.cc:5840] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c4fe5e44448d98db1eed6329d150d (127.0.72.130). New cstate: current_term: 1 leader_uuid: "245c4fe5e44448d98db1eed6329d150d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: UNKNOWN } } }
I20261118 14:58:14.202371 601 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
I20261118 14:58:14.202693 601 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:14.219244 601 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7] alter complete (version 1)
I20261118 14:58:14.220643 601 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]). Expected version 1 got 0
I20261118 14:58:14.221298 601 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60882:
name: "default.Table_4d2c8b2bcb994782a8bbf4b38ca2f7af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:14.235977 917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87ee223b5a246c58e9b2eca5740d08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:14.247213 1670 consensus_queue.cc:1049] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:14.249814 1655 tablet_bootstrap.cc:492] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:14.251664 1655 tablet_bootstrap.cc:654] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:14.252763 1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87ee223b5a246c58e9b2eca5740d08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:14.260741 1654 tablet.cc:1759] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:14.261566 1652 tablet.cc:1759] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:14.285228 1658 tablet.cc:1759] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:14.288189 1636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:14.288424 1636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:14.296175 1655 tablet_bootstrap.cc:492] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:14.300806 1655 ts_tablet_manager.cc:1449] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.051s user 0.013s sys 0.000s
I20261118 14:58:14.301885 1655 raft_consensus.cc:360] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.303194 1655 raft_consensus.cc:386] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:14.303452 1655 raft_consensus.cc:748] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:14.303931 1655 consensus_queue.cc:261] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.306248 1655 ts_tablet_manager.cc:1480] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20261118 14:58:14.329643 1633 ts_tablet_manager.cc:1975] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.329813 1633 log.cc:1199] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/92fab0337a8e4ffeb604589ac588b0af
I20261118 14:58:14.330226 1633 ts_tablet_manager.cc:1996] T 92fab0337a8e4ffeb604589ac588b0af P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:14.334990 1634 ts_tablet_manager.cc:1975] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.335106 1633 tablet_replica.cc:333] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:14.335166 1634 log.cc:1199] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/91a4a7f508b44f809c8071413df8d648
I20261118 14:58:14.335297 1633 raft_consensus.cc:2261] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:14.335572 1634 ts_tablet_manager.cc:1996] T 91a4a7f508b44f809c8071413df8d648 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:14.335671 1633 raft_consensus.cc:2290] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:14.336944 587 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 92fab0337a8e4ffeb604589ac588b0af (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]) successfully deleted
I20261118 14:58:14.337411 1633 ts_tablet_manager.cc:1962] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:14.338630 1637 ts_tablet_manager.cc:1975] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.338788 1637 log.cc:1199] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/91a4a7f508b44f809c8071413df8d648
I20261118 14:58:14.339182 1637 ts_tablet_manager.cc:1996] T 91a4a7f508b44f809c8071413df8d648 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:14.348917 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87ee223b5a246c58e9b2eca5740d08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:14.359771 1665 tablet_bootstrap.cc:492] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:14.361197 1653 tablet_bootstrap.cc:492] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:14.361600 1665 tablet_bootstrap.cc:654] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:14.362905 1653 tablet_bootstrap.cc:654] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:14.380221 1665 tablet_bootstrap.cc:492] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:14.380419 1665 ts_tablet_manager.cc:1449] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.018s
I20261118 14:58:14.381242 1665 raft_consensus.cc:360] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.381448 1665 raft_consensus.cc:386] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:14.381533 1665 raft_consensus.cc:748] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:14.382010 1665 consensus_queue.cc:261] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.383934 1665 ts_tablet_manager.cc:1480] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20261118 14:58:14.385272 1665 tablet_bootstrap.cc:492] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:14.387890 1665 tablet_bootstrap.cc:654] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:14.389489 1653 tablet_bootstrap.cc:492] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:14.389664 1653 ts_tablet_manager.cc:1449] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.004s
I20261118 14:58:14.390481 1653 raft_consensus.cc:360] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.390672 1653 raft_consensus.cc:386] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:14.390760 1653 raft_consensus.cc:748] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:14.391242 1653 consensus_queue.cc:261] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.393635 1653 ts_tablet_manager.cc:1480] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20261118 14:58:14.396653 1665 tablet_bootstrap.cc:492] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:14.396857 1665 ts_tablet_manager.cc:1449] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.003s
I20261118 14:58:14.397655 1665 raft_consensus.cc:360] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.397925 1665 raft_consensus.cc:386] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:14.398056 1665 raft_consensus.cc:748] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:14.398394 1665 consensus_queue.cc:261] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.399946 1665 ts_tablet_manager.cc:1480] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20261118 14:58:14.424289 1145 consensus_peers.cc:597] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Couldn't send request to peer 7c53fe91804746e8bfcf743e0ab8ba6f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cdc9a8de0308404fb2fa23b70984f12b. This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:14.431350 1637 tablet_replica.cc:333] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:14.431567 1637 raft_consensus.cc:2261] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:14.431582 590 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 91a4a7f508b44f809c8071413df8d648 (table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]) successfully deleted
I20261118 14:58:14.431835 1637 raft_consensus.cc:2290] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:14.432442 1637 ts_tablet_manager.cc:1962] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:14.434962 1609 raft_consensus.cc:499] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:14.435128 1609 raft_consensus.cc:521] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.435256 1633 ts_tablet_manager.cc:1975] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:14.435359 1633 log.cc:1199] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/f4bdcca79fcb4fedaea1fdbe2dd31112
I20261118 14:58:14.435763 1633 ts_tablet_manager.cc:1996] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:14.435863 1609 leader_election.cc:290] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:14.438436 1634 tablet_replica.cc:333] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:14.438623 1634 raft_consensus.cc:2261] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:14.438992 1634 raft_consensus.cc:2290] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:14.439548 1634 ts_tablet_manager.cc:1962] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:14.439600 589 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 91a4a7f508b44f809c8071413df8d648 (table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]) successfully deleted
I20261118 14:58:14.441468 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "212e4a39f2d0465a854b3474dc020aa6" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:14.441648 1211 raft_consensus.cc:2490] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:14.441983 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "212e4a39f2d0465a854b3474dc020aa6" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:14.442190 1076 raft_consensus.cc:2490] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:14.442559 874 leader_election.cc:304] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:14.442874 1610 raft_consensus.cc:2826] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:14.442965 1610 raft_consensus.cc:499] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:14.443054 1610 raft_consensus.cc:3096] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.500327 587 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:14.500298 1633 tablet_replica.cc:333] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:14.500793 1633 raft_consensus.cc:2261] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:14.501096 1633 raft_consensus.cc:2290] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:14.501667 1633 ts_tablet_manager.cc:1962] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:14.536515 1610 raft_consensus.cc:521] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.537321 1610 leader_election.cc:290] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:14.537847 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "212e4a39f2d0465a854b3474dc020aa6" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:14.538043 1076 raft_consensus.cc:3096] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.540978 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "212e4a39f2d0465a854b3474dc020aa6" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:14.541198 1211 raft_consensus.cc:3096] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.544926 1668 raft_consensus.cc:499] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:14.545095 1668 raft_consensus.cc:521] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.545753 1668 leader_election.cc:290] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:14.546171 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a00587922d884ced8250ac130789da68" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:14.546231 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a00587922d884ced8250ac130789da68" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:14.546382 938 raft_consensus.cc:2490] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:14.546450 1210 raft_consensus.cc:2490] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:14.546808 1010 leader_election.cc:304] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:14.547098 1668 raft_consensus.cc:2826] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:14.547197 1668 raft_consensus.cc:499] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:14.547297 1668 raft_consensus.cc:3096] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.575070 1608 raft_consensus.cc:499] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:14.575228 1608 raft_consensus.cc:521] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.575807 1608 leader_election.cc:290] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:14.576241 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1b82ed8e1b7244e7b28fed1b93bec69c" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:14.576298 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1b82ed8e1b7244e7b28fed1b93bec69c" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:14.576431 938 raft_consensus.cc:2490] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:14.576469 1210 raft_consensus.cc:2490] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:14.576831 1010 leader_election.cc:304] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:14.577095 1608 raft_consensus.cc:2826] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:14.577184 1608 raft_consensus.cc:499] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:14.577282 1608 raft_consensus.cc:3096] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.626973 1369 raft_consensus.cc:499] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:14.627156 1369 raft_consensus.cc:521] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.627714 1369 leader_election.cc:290] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:14.628075 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f87ee223b5a246c58e9b2eca5740d08c" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:14.628206 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f87ee223b5a246c58e9b2eca5740d08c" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:14.628276 938 raft_consensus.cc:2490] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:14.628391 1075 raft_consensus.cc:2490] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:14.628720 1145 leader_election.cc:304] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:14.629014 1369 raft_consensus.cc:2826] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:14.629110 1369 raft_consensus.cc:499] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:14.629213 1369 raft_consensus.cc:3096] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.638564 1211 raft_consensus.cc:2490] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:14.638563 1076 raft_consensus.cc:2490] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:14.638652 1637 ts_tablet_manager.cc:1975] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:14.638778 1637 log.cc:1199] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/f4bdcca79fcb4fedaea1fdbe2dd31112
I20261118 14:58:14.639014 875 leader_election.cc:304] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:14.639154 1637 ts_tablet_manager.cc:1996] T f4bdcca79fcb4fedaea1fdbe2dd31112 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:14.639312 1610 raft_consensus.cc:2826] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:14.639443 1610 raft_consensus.cc:705] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:14.639536 1633 ts_tablet_manager.cc:1975] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.639672 1633 log.cc:1199] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/91a4a7f508b44f809c8071413df8d648
I20261118 14:58:14.639695 1610 consensus_queue.cc:238] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.640014 1633 ts_tablet_manager.cc:1996] T 91a4a7f508b44f809c8071413df8d648 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:14.640714 1634 ts_tablet_manager.cc:1975] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.640820 1634 log.cc:1199] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/cdc9a8de0308404fb2fa23b70984f12b
I20261118 14:58:14.641193 1634 ts_tablet_manager.cc:1996] T cdc9a8de0308404fb2fa23b70984f12b P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:14.641284 1369 raft_consensus.cc:521] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.642735 1369 leader_election.cc:290] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:14.643918 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f87ee223b5a246c58e9b2eca5740d08c" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:14.644088 938 raft_consensus.cc:3096] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.644698 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f87ee223b5a246c58e9b2eca5740d08c" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:14.644865 1075 raft_consensus.cc:3096] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.644948 1610 consensus_queue.cc:1049] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:14.645246 1682 consensus_queue.cc:1049] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 14:58:14.725167 1684 log.cc:927] Time spent T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
W20261118 14:58:14.725167 1681 log.cc:927] Time spent T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
W20261118 14:58:14.725167 1683 log.cc:927] Time spent T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
I20261118 14:58:14.726426 1634 tablet_replica.cc:333] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:14.726624 1634 raft_consensus.cc:2261] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:14.726796 1634 raft_consensus.cc:2290] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:14.727232 587 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 91a4a7f508b44f809c8071413df8d648 (table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]) successfully deleted
I20261118 14:58:14.727337 1634 ts_tablet_manager.cc:1962] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:14.727964 589 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet cdc9a8de0308404fb2fa23b70984f12b (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:14.728897 1637 tablet_replica.cc:333] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:14.729043 590 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:14.729102 1637 raft_consensus.cc:2261] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:14.729262 1637 raft_consensus.cc:2290] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:14.729802 1637 ts_tablet_manager.cc:1962] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:14.739370 1668 raft_consensus.cc:521] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.740046 1668 leader_election.cc:290] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:14.740595 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a00587922d884ced8250ac130789da68" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:14.740790 1211 raft_consensus.cc:3096] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.741845 1075 raft_consensus.cc:2490] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:14.742031 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:33967
--webserver_interface=127.0.72.188
--webserver_port=42503
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 14:58:14.743603 1148 leader_election.cc:304] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:14.743587 938 raft_consensus.cc:2490] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:14.743891 1369 raft_consensus.cc:2826] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:14.748286 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a00587922d884ced8250ac130789da68" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:14.748565 938 raft_consensus.cc:3096] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.751827 1369 raft_consensus.cc:705] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:14.752621 1369 consensus_queue.cc:238] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.764276 1685 consensus_queue.cc:1049] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:14.764971 1685 consensus_queue.cc:1049] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:14.835594 1608 raft_consensus.cc:521] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.835757 938 raft_consensus.cc:2490] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:14.836256 1211 raft_consensus.cc:2490] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:14.836745 1634 ts_tablet_manager.cc:1975] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:14.836776 1608 leader_election.cc:290] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:14.836882 1634 log.cc:1199] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/f4bdcca79fcb4fedaea1fdbe2dd31112
I20261118 14:58:14.837208 1637 ts_tablet_manager.cc:1975] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:14.837266 1634 ts_tablet_manager.cc:1996] T f4bdcca79fcb4fedaea1fdbe2dd31112 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:14.837323 1637 log.cc:1199] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/cdc9a8de0308404fb2fa23b70984f12b
I20261118 14:58:14.837661 1637 ts_tablet_manager.cc:1996] T cdc9a8de0308404fb2fa23b70984f12b P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:14.837882 1012 leader_election.cc:304] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:14.838191 1608 raft_consensus.cc:2826] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:14.838349 1608 raft_consensus.cc:705] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:14.838435 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1b82ed8e1b7244e7b28fed1b93bec69c" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:14.838599 1211 raft_consensus.cc:3096] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.838629 1608 consensus_queue.cc:238] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.839816 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1b82ed8e1b7244e7b28fed1b93bec69c" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:14.840049 938 raft_consensus.cc:3096] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:14.842895 1668 consensus_queue.cc:1049] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:14.843631 1668 consensus_queue.cc:1049] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:14.860342 589 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet f4bdcca79fcb4fedaea1fdbe2dd31112 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:14.896195 590 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet cdc9a8de0308404fb2fa23b70984f12b (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:14.936908 1211 raft_consensus.cc:2490] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:14.936908 938 raft_consensus.cc:2490] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:14.937638 1012 leader_election.cc:304] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:14.938028 1670 raft_consensus.cc:2826] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:14.938220 1670 raft_consensus.cc:705] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:14.938561 1670 consensus_queue.cc:238] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:14.941612 1608 consensus_queue.cc:1049] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:14.941967 1668 consensus_queue.cc:1049] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 14:58:15.085219 587 catalog_manager.cc:4847] Async tablet task AlterTable RPC for tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a00587922d884ced8250ac130789da68 consensus state has no leader
W20261118 14:58:15.159703 1686 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:15.160048 1686 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:15.160156 1686 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:15.168830 1686 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:15.168924 1686 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:15.168991 1686 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:15.169066 1686 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:15.182498 1686 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.72.188
--webserver_port=42503
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:15.184875 1686 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:15.187389 1686 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:15.195998 1703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:15.196702 1704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:15.197094 1706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:15.197574 1686 server_base.cc:1061] running on GCE node
I20261118 14:58:15.198292 1686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:15.199960 1686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:15.201169 1686 hybrid_clock.cc:648] HybridClock initialized: now 1795013895201074 us; error 82 us; skew 500 ppm
I20261118 14:58:15.201575 1686 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:15.204159 1686 webserver.cc:577] Webserver started at http://127.0.72.188:42503/ using document root <none> and password file <none>
I20261118 14:58:15.206229 1686 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:15.206399 1686 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:15.218263 1686 fs_manager.cc:714] Time spent opening directory manager: real 0.009s user 0.003s sys 0.006s
I20261118 14:58:15.236799 1712 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:15.238682 1686 fs_manager.cc:730] Time spent opening block manager: real 0.019s user 0.005s sys 0.004s
I20261118 14:58:15.238860 1686 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:15.239614 1686 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:15.261483 1686 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:15.262393 1686 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:15.262691 1686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:15.298683 1686 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.188:33967
I20261118 14:58:15.298738 1765 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.188:33967 every 8 connection(s)
I20261118 14:58:15.300407 1686 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
I20261118 14:58:15.302734 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 1686
I20261118 14:58:15.309547 1766 sys_catalog.cc:263] Verifying existing consensus state
I20261118 14:58:15.315357 1766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap starting.
I20261118 14:58:15.332808 1766 log.cc:826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:15.377136 1766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=57 overwritten=0 applied=57 ignored=0} inserts{seen=37 ignored=0} mutations{seen=42 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 14:58:15.377673 1766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap complete.
I20261118 14:58:15.383348 1766 raft_consensus.cc:360] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:15.384179 1766 raft_consensus.cc:748] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b90720bc0b34e4994abf4fc6f9e59e1, State: Initialized, Role: FOLLOWER
I20261118 14:58:15.384902 1766 consensus_queue.cc:261] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 57, Last appended: 2.57, Last appended by leader: 57, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:15.385591 1770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:15.385815 1770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:15.386714 1766 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:15.404968 1784 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:15.405097 1784 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: loading cluster ID for follower catalog manager: success
I20261118 14:58:15.410117 1784 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: acquiring CA information for follower catalog manager: success
I20261118 14:58:15.412932 1784 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:15.481336 984 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:15.483088 1257 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:15.486817 1729 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" instance_seqno: 1795013887687621) as {username='slave'} at 127.0.72.129:48783; Asking this server to re-register.
I20261118 14:58:15.488538 1729 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" instance_seqno: 1795013888668865) as {username='slave'} at 127.0.72.131:53823; Asking this server to re-register.
I20261118 14:58:15.489334 1257 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:15.490252 1122 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:15.490444 984 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:15.491756 1728 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:15.492118 1729 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:15.494828 1728 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" instance_seqno: 1795013888201158) as {username='slave'} at 127.0.72.130:53775; Asking this server to re-register.
I20261118 14:58:15.495493 1122 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:15.496876 1728 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:15.597826 1794 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:15.598039 1794 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:15.598785 1794 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:15.599308 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 3 candidate_status { last_received { term: 2 index: 71 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e" is_pre_election: true
I20261118 14:58:15.605515 1739 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 3 candidate_status { last_received { term: 2 index: 71 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:15.606154 1739 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9ef96ff074450a879e66787486386d in term 2.
I20261118 14:58:15.606760 1465 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e
I20261118 14:58:15.607062 1794 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20261118 14:58:15.607167 1794 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:15.607255 1794 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 2 FOLLOWER]: Advancing to term 3
I20261118 14:58:15.632915 1794 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:15.633780 1794 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 3 election: Requested vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:15.634390 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 3 candidate_status { last_received { term: 2 index: 71 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e"
I20261118 14:58:15.634654 1739 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 3 candidate_status { last_received { term: 2 index: 71 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
I20261118 14:58:15.634899 1739 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 2 FOLLOWER]: Advancing to term 3
I20261118 14:58:15.640120 1488 raft_consensus.cc:1250] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 FOLLOWER]: Rejecting Update request from peer 0ed65a3719634e519d783540afd6189e for earlier term 2. Current term is 3. Ops: []
I20261118 14:58:15.640810 1636 consensus_queue.cc:1060] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 }, Status: INVALID_TERM, Last received: 2.71, Next index: 72, Last known committed idx: 71, Time since last communication: 0.000s
I20261118 14:58:15.641337 1636 raft_consensus.cc:3091] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 2 LEADER]: Stepping down as leader of term 2
I20261118 14:58:15.641448 1636 raft_consensus.cc:748] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ed65a3719634e519d783540afd6189e, State: Running, Role: LEADER
I20261118 14:58:15.641758 1636 consensus_queue.cc:261] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 71, Committed index: 71, Last appended: 2.71, Last appended by leader: 71, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:15.642205 1636 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 2 FOLLOWER]: Advancing to term 3
I20261118 14:58:15.738677 1739 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9ef96ff074450a879e66787486386d in term 3.
I20261118 14:58:15.739415 1465 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e
I20261118 14:58:15.739847 1794 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 FOLLOWER]: Leader election won for term 3
I20261118 14:58:15.741518 1794 raft_consensus.cc:705] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 LEADER]: Becoming Leader. State: Replica: 7c9ef96ff074450a879e66787486386d, State: Running, Role: LEADER
I20261118 14:58:15.742022 1794 consensus_queue.cc:238] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 71, Committed index: 71, Last appended: 2.71, Last appended by leader: 71, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:15.747442 613 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 3 FOLLOWER]: Refusing update from remote peer 7c9ef96ff074450a879e66787486386d: Log matching property violated. Preceding OpId in replica: term: 2 index: 71. Preceding OpId from leader: term: 3 index: 72. (index mismatch)
I20261118 14:58:15.747517 1739 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 3 FOLLOWER]: Refusing update from remote peer 7c9ef96ff074450a879e66787486386d: Log matching property violated. Preceding OpId in replica: term: 2 index: 57. Preceding OpId from leader: term: 3 index: 72. (index mismatch)
I20261118 14:58:15.748379 1799 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 71, Time since last communication: 0.001s
I20261118 14:58:15.756918 1799 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 72, Last known committed idx: 57, Time since last communication: 0.000s
I20261118 14:58:15.757441 1797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c9ef96ff074450a879e66787486386d. Latest consensus state: current_term: 3 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:15.757881 1797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:15.762991 1802 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 14:58:15.770725 1802 catalog_manager.cc:694] Loaded metadata for table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:15.771328 1802 catalog_manager.cc:694] Loaded metadata for table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]
I20261118 14:58:15.771905 1802 catalog_manager.cc:694] Loaded metadata for table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:15.772395 1802 catalog_manager.cc:694] Loaded metadata for table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:15.772856 1802 catalog_manager.cc:694] Loaded metadata for table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]
I20261118 14:58:15.773288 1802 catalog_manager.cc:694] Loaded metadata for table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]
I20261118 14:58:15.773667 1802 catalog_manager.cc:694] Loaded metadata for table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:15.774015 1802 catalog_manager.cc:694] Loaded metadata for table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:15.774938 1802 catalog_manager.cc:694] Loaded metadata for table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]
I20261118 14:58:15.775368 1802 catalog_manager.cc:694] Loaded metadata for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:15.775790 1802 catalog_manager.cc:694] Loaded metadata for table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]
I20261118 14:58:15.776187 1802 catalog_manager.cc:694] Loaded metadata for table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]
I20261118 14:58:15.776629 1802 catalog_manager.cc:694] Loaded metadata for table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]
I20261118 14:58:15.778884 1636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c9ef96ff074450a879e66787486386d. Latest consensus state: current_term: 3 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:15.779116 1636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:15.779924 1802 tablet_loader.cc:96] loaded metadata for tablet 1b82ed8e1b7244e7b28fed1b93bec69c (table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985])
I20261118 14:58:15.786386 1802 tablet_loader.cc:96] loaded metadata for tablet 212e4a39f2d0465a854b3474dc020aa6 (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7])
I20261118 14:58:15.786901 1802 tablet_loader.cc:96] loaded metadata for tablet 309f067161fe469dbb29d14c16139058 (table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15])
I20261118 14:58:15.787603 1802 tablet_loader.cc:96] loaded metadata for tablet 75581812cbcb48e5a2ea9d0fb925c03a (table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe])
I20261118 14:58:15.787662 1800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:15.788095 1802 tablet_loader.cc:96] loaded metadata for tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b])
I20261118 14:58:15.788136 1800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:15.788482 1802 tablet_loader.cc:96] loaded metadata for tablet 8a07aac51e8a48dba2c61e9e477fad11 (table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f])
I20261118 14:58:15.789891 1802 tablet_loader.cc:96] loaded metadata for tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0])
I20261118 14:58:15.790423 1636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:15.790661 1636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:15.790657 1802 tablet_loader.cc:96] loaded metadata for tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d])
I20261118 14:58:15.791275 1802 tablet_loader.cc:96] loaded metadata for tablet cf7800d54792490faa3860de0ecb7a3a (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7])
I20261118 14:58:15.791929 1802 tablet_loader.cc:96] loaded metadata for tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6])
I20261118 14:58:15.792652 1802 tablet_loader.cc:96] loaded metadata for tablet e47d7d10a5c741ecb30ec701c76b27a3 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f])
I20261118 14:58:15.793303 1802 tablet_loader.cc:96] loaded metadata for tablet f66c4e0f06f043108a6451666a80a5f9 (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28])
I20261118 14:58:15.793731 1802 tablet_loader.cc:96] loaded metadata for tablet f87ee223b5a246c58e9b2eca5740d08c (table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627])
I20261118 14:58:15.794090 1802 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:15.795742 1802 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:15.795814 1802 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:15.799264 1802 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:15.800973 1802 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Loaded TSK: 0
I20261118 14:58:15.802603 1802 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:15.804839 1802 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:15.807075 1802 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 19
I20261118 14:58:15.808475 1770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c9ef96ff074450a879e66787486386d. Latest consensus state: current_term: 3 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:15.808768 1770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:15.810285 1800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:15.810519 1800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:15.813428 1770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:15.813642 1770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
W20261118 14:58:15.855947 823 hms_client.cc:355] Time spent drop HMS table: real 1.711s user 0.000s sys 0.000s
I20261118 14:58:15.856168 603 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]
I20261118 14:58:15.917551 599 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]
I20261118 14:58:15.974020 600 catalog_manager.cc:3543] renamed table default.table_537f6e87a8d040a695fda278ff06cbf5 [id=07fa88e7fcd044fca002cc02a8788a4d] in HMS: new name default.table_277b8485c5ba4cdfaa228990cb8a41ed
I20261118 14:58:16.006192 594 catalog_manager.cc:2215] created HMS catalog entry for table [id=7254e5f1dc9a48a2a6fa1f0ed42fd295]
W20261118 14:58:16.007180 594 catalog_manager.cc:2243] Illegal state: an error occurred while writing to the sys-catalog: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER
I20261118 14:58:16.028844 598 catalog_manager.cc:2215] created HMS catalog entry for table [id=8cc4e4385a0b474baca66e94b8a7346d]
W20261118 14:58:16.030865 598 catalog_manager.cc:2243] Illegal state: an error occurred while writing to the sys-catalog: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER
I20261118 14:58:16.051100 595 catalog_manager.cc:2215] created HMS catalog entry for table [id=c32b621587d1449d8931a348b7f8b654]
W20261118 14:58:16.051893 595 catalog_manager.cc:2243] Illegal state: an error occurred while writing to the sys-catalog: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER
I20261118 14:58:16.085945 596 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]
I20261118 14:58:16.105472 597 catalog_manager.cc:2215] created HMS catalog entry for table [id=0ce3ba58cd7440a9b27427e7713af331]
W20261118 14:58:16.106261 597 catalog_manager.cc:2243] Illegal state: an error occurred while writing to the sys-catalog: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER
I20261118 14:58:16.126899 602 catalog_manager.cc:2215] created HMS catalog entry for table [id=378bd168f216477992c1b7bb33791778]
W20261118 14:58:16.127888 602 catalog_manager.cc:2243] Illegal state: an error occurred while writing to the sys-catalog: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER
W20261118 14:58:16.131635 826 hms_notification_log_listener.cc:143] Hive Metastore notification log listener poll failed: Service unavailable: lost leadership while handling Hive Metastore notification log events
I20261118 14:58:16.132189 603 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60882 (request call id 13) took 2308 ms. Trace:
I20261118 14:58:16.132407 599 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60882 (request call id 17) took 2300 ms. Trace:
I20261118 14:58:16.132345 603 rpcz_store.cc:276] 1118 14:58:13.823190 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.829685 (+ 6495us) service_pool.cc:224] Handling call
1118 14:58:14.142280 (+312595us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 14:58:16.132107 (+1989827us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":2241,"Hive Metastore.run_cpu_time_us":474,"Hive Metastore.run_wall_time_us":1711312}
I20261118 14:58:16.133574 600 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60882 (request call id 5) took 2311 ms. Trace:
I20261118 14:58:16.133683 600 rpcz_store.cc:276] 1118 14:58:13.822274 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.827111 (+ 4837us) service_pool.cc:224] Handling call
1118 14:58:14.142733 (+315622us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 14:58:16.132786 (+1990053us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 14:58:16.133170 (+ 384us) catalog_manager.cc:3766] Apply alter schema
1118 14:58:16.133515 (+ 345us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1774551,"Hive Metastore.run_cpu_time_us":1087,"Hive Metastore.run_wall_time_us":56425}
W20261118 14:58:16.134006 603 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.130:32931 (request call id 61) took 1912 ms (client timeout 1000 ms). Trace:
W20261118 14:58:16.134111 603 rpcz_store.cc:269] 1118 14:58:14.221818 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.133763 (+1911945us) service_pool.cc:211] Skipping call since client already timed out
1118 14:58:16.133935 (+ 172us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20261118 14:58:16.132261 601 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 18) took 2300 ms. Trace:
I20261118 14:58:16.134284 601 rpcz_store.cc:276] 1118 14:58:13.831806 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:14.221097 (+389291us) service_pool.cc:224] Handling call
1118 14:58:16.132207 (+1911110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.134776 596 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60882 (request call id 15) took 2311 ms. Trace:
I20261118 14:58:16.134860 596 rpcz_store.cc:276] 1118 14:58:13.823477 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.849452 (+ 25975us) service_pool.cc:224] Handling call
1118 14:58:13.888412 (+ 38960us) spinlock_profiling.cc:245] Waited 38.7 ms on lock 0x616000043354. stack: 00007f26cab673a6 00007f26cab67160 00007f26c9b5834d 00007f26e112b704 00007f26e112b6cd 00007f26da07d82d 00007f26da07d923 00007f26d6be7221 00007f26deadc50d 00007f26dec6c866 00007f26d55761e6 00007f26d5575f15 00007f26d1caf8de 00007f26d1cae7be 00007f26d1cb49f8 00007f26d1cb535f
1118 14:58:14.144508 (+256096us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 14:58:16.134719 (+1990211us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1906390,"Hive Metastore.run_cpu_time_us":256,"Hive Metastore.run_wall_time_us":34849,"spinlock_wait_cycles":88957184}
W20261118 14:58:16.135865 601 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.130:32931 (request call id 62) took 910 ms (client timeout 1000 ms). Trace:
W20261118 14:58:16.135962 601 rpcz_store.cc:269] 1118 14:58:15.225314 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.134473 (+909159us) service_pool.cc:224] Handling call
1118 14:58:16.135392 (+ 919us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 14:58:16.136335 596 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.129:58169 (request call id 72) took 889 ms (client timeout 1000 ms). Trace:
W20261118 14:58:16.136426 596 rpcz_store.cc:269] 1118 14:58:15.246532 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.135077 (+888545us) service_pool.cc:224] Handling call
1118 14:58:16.136277 (+ 1200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.136890 596 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60882 (request call id 19) took 2304 ms. Trace:
I20261118 14:58:16.136977 596 rpcz_store.cc:276] 1118 14:58:13.831883 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.136593 (+2304710us) service_pool.cc:224] Handling call
1118 14:58:16.136835 (+ 242us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 14:58:16.137091 601 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.131:57689 (request call id 69) took 887 ms (client timeout 1000 ms). Trace:
W20261118 14:58:16.137171 601 rpcz_store.cc:269] 1118 14:58:15.249926 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.136172 (+886246us) service_pool.cc:224] Handling call
1118 14:58:16.137040 (+ 868us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.137472 596 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 20) took 2305 ms. Trace:
I20261118 14:58:16.137558 596 rpcz_store.cc:276] 1118 14:58:13.831973 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.137121 (+2305148us) service_pool.cc:224] Handling call
1118 14:58:16.137420 (+ 299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.138067 596 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60882 (request call id 22) took 2305 ms. Trace:
I20261118 14:58:16.138149 596 rpcz_store.cc:276] 1118 14:58:13.832166 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.137746 (+2305580us) service_pool.cc:224] Handling call
1118 14:58:16.138015 (+ 269us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 14:58:16.139091 1264 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.190:42447)
W20261118 14:58:16.139525 1264 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.190:42447)
W20261118 14:58:16.134025 600 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.129:58169 (request call id 71) took 1889 ms (client timeout 1000 ms). Trace:
W20261118 14:58:16.140069 600 rpcz_store.cc:269] 1118 14:58:14.244332 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.133935 (+1889603us) service_pool.cc:211] Skipping call since client already timed out
1118 14:58:16.133979 (+ 44us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20261118 14:58:16.140539 600 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60882 (request call id 24) took 2308 ms. Trace:
I20261118 14:58:16.140630 600 rpcz_store.cc:276] 1118 14:58:13.832342 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.140228 (+2307886us) service_pool.cc:224] Handling call
1118 14:58:16.140495 (+ 267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.141052 600 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60882 (request call id 25) took 2307 ms. Trace:
I20261118 14:58:16.141129 600 rpcz_store.cc:276] 1118 14:58:13.833923 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.140769 (+2306846us) service_pool.cc:224] Handling call
1118 14:58:16.141005 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.141539 600 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:60882 (request call id 26) took 2307 ms. Trace:
I20261118 14:58:16.141633 600 rpcz_store.cc:276] 1118 14:58:13.834067 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.141268 (+2307201us) service_pool.cc:224] Handling call
1118 14:58:16.141470 (+ 202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 14:58:16.141844 587 catalog_manager.cc:4847] Async tablet task AlterTable RPC for tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a00587922d884ced8250ac130789da68 consensus state has no leader
I20261118 14:58:16.142062 600 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 27) took 2307 ms. Trace:
I20261118 14:58:16.142138 600 rpcz_store.cc:276] 1118 14:58:13.834167 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.141774 (+2307607us) service_pool.cc:224] Handling call
1118 14:58:16.142018 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 14:58:16.145835 1264 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.189:46707)
I20261118 14:58:16.145831 1479 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_ced554c65cd4465d9ddab052dc32efcf" } modify_external_catalogs: true
I20261118 14:58:16.146085 1479 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_ced554c65cd4465d9ddab052dc32efcf" } modify_external_catalogs: true
I20261118 14:58:16.146786 1478 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_7171b2f0b6ea470393bf9aa28246cf65"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:16.147444 1477 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_4d2c8b2bcb994782a8bbf4b38ca2f7af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20261118 14:58:16.148103 603 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.72.131:57689 (request call id 68) took 1901 ms (client timeout 1000 ms). Trace:
W20261118 14:58:16.148195 603 rpcz_store.cc:269] 1118 14:58:14.246489 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.134319 (+1887830us) service_pool.cc:211] Skipping call since client already timed out
1118 14:58:16.134350 (+ 31us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20261118 14:58:16.148452 603 master_service.cc:965] ReplaceTablet: received request to replace tablet 212e4a39f2d0465a854b3474dc020aa6 from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:16.148789 603 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60882 (request call id 29) took 1994 ms. Trace:
I20261118 14:58:16.148888 603 rpcz_store.cc:276] 1118 14:58:14.154631 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.148357 (+1993726us) service_pool.cc:224] Handling call
1118 14:58:16.148758 (+ 401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.149112 603 master_service.cc:965] ReplaceTablet: received request to replace tablet a00587922d884ced8250ac130789da68 from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:16.149401 603 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60882 (request call id 30) took 1994 ms. Trace:
I20261118 14:58:16.149477 603 rpcz_store.cc:276] 1118 14:58:14.154746 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.149063 (+1994317us) service_pool.cc:224] Handling call
1118 14:58:16.149349 (+ 286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.149698 603 master_service.cc:965] ReplaceTablet: received request to replace tablet f4bdcca79fcb4fedaea1fdbe2dd31112 from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:16.149992 603 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60882 (request call id 31) took 1993 ms. Trace:
I20261118 14:58:16.150091 603 rpcz_store.cc:276] 1118 14:58:14.157016 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.149623 (+1992607us) service_pool.cc:224] Handling call
1118 14:58:16.149961 (+ 338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.150303 603 master_service.cc:965] ReplaceTablet: received request to replace tablet a00587922d884ced8250ac130789da68 from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:16.153251 1474 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_59239feaba874d6890fe8e86d57545e2" } new_table_name: "default.Table_a69dc21b9f4948ffad3423d68bff8f34" modify_external_catalogs: true
I20261118 14:58:16.153317 1473 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_38deaf819a5e430090d96b02cbd9cdc3" } new_table_name: "default.Table_3f41e3f42a794945838124ce6712ceb3" modify_external_catalogs: true
I20261118 14:58:16.137580 601 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:60882 (request call id 21) took 2305 ms. Trace:
I20261118 14:58:16.153859 601 rpcz_store.cc:276] 1118 14:58:13.832080 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.137313 (+2305233us) service_pool.cc:224] Handling call
1118 14:58:16.137527 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.154086 1476 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_7886615812a14a0492f39f2694999785" } new_table_name: "default.Table_927d6a72d3524190964efb6e9f1f8d42" modify_external_catalogs: true
I20261118 14:58:16.151245 596 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 23) took 2318 ms. Trace:
I20261118 14:58:16.154379 596 rpcz_store.cc:276] 1118 14:58:13.832257 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.138306 (+2306049us) service_pool.cc:224] Handling call
1118 14:58:16.151198 (+ 12892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.151304 600 master_service.cc:965] ReplaceTablet: received request to replace tablet cf7800d54792490faa3860de0ecb7a3a from {username='slave'} at 127.0.0.1:60882
I20261118 14:58:16.156615 600 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60882 (request call id 28) took 2007 ms. Trace:
I20261118 14:58:16.156790 600 rpcz_store.cc:276] 1118 14:58:14.149086 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.142283 (+1993197us) service_pool.cc:224] Handling call
1118 14:58:16.156567 (+ 14284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.154783 601 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 33) took 1796 ms. Trace:
I20261118 14:58:16.157166 601 rpcz_store.cc:276] 1118 14:58:14.358187 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.154096 (+1795909us) service_pool.cc:224] Handling call
1118 14:58:16.154730 (+ 634us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.157837 603 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:60882 (request call id 32) took 1993 ms. Trace:
I20261118 14:58:16.157970 603 rpcz_store.cc:276] 1118 14:58:14.164266 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:16.150253 (+1985987us) service_pool.cc:224] Handling call
1118 14:58:16.157780 (+ 7527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 14:58:16.158583 1475 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_ab0fd883cbec4e0b8af7febddffe78c5" } new_table_name: "default.Table_2f1acae65de24ae9a7a19515d10acf9c" modify_external_catalogs: true
I20261118 14:58:16.159065 1472 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_9021a49aff8546eca95c57bde20822b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20261118 14:58:16.161813 1264 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.189:46707)
I20261118 14:58:16.167577 1531 catalog_manager.cc:2813] Deleting table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe] in response to Hive Metastore notification log event 21
I20261118 14:58:16.170518 1470 master_service.cc:965] ReplaceTablet: received request to replace tablet a00587922d884ced8250ac130789da68 from {username='slave'} at 127.0.0.1:50956
I20261118 14:58:16.132472 599 rpcz_store.cc:276] 1118 14:58:13.831724 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.886514 (+ 54790us) service_pool.cc:224] Handling call
1118 14:58:14.142572 (+256058us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 14:58:16.132358 (+1989786us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1713445,"Hive Metastore.run_cpu_time_us":250,"Hive Metastore.run_wall_time_us":61281}
I20261118 14:58:16.167537 1471 master_service.cc:965] ReplaceTablet: received request to replace tablet 212e4a39f2d0465a854b3474dc020aa6 from {username='slave'} at 127.0.0.1:50956
I20261118 14:58:16.202270 1531 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet 75581812cbcb48e5a2ea9d0fb925c03a
I20261118 14:58:16.202270 1470 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet a00587922d884ced8250ac130789da68
I20261118 14:58:16.209177 1531 catalog_manager.cc:2813] Deleting table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985] in response to Hive Metastore notification log event 22
I20261118 14:58:16.213186 1471 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet 212e4a39f2d0465a854b3474dc020aa6
I20261118 14:58:16.213697 1471 catalog_manager.cc:6936] ReplaceTablet: tablet 212e4a39f2d0465a854b3474dc020aa6 deleted and replaced by tablet 5335a58da46b48019aec4b66e745fe68
I20261118 14:58:16.214069 1471 master_service.cc:965] ReplaceTablet: received request to replace tablet f4bdcca79fcb4fedaea1fdbe2dd31112 from {username='slave'} at 127.0.0.1:50956
I20261118 14:58:16.214855 1471 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:16.215087 1471 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:16.228096 1470 catalog_manager.cc:6936] ReplaceTablet: tablet a00587922d884ced8250ac130789da68 deleted and replaced by tablet 522efb108dd140b8a3fd476d202a073e
I20261118 14:58:16.228483 1470 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_155ca12a699d4393b30979580a04864a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:16.248469 1531 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet 1b82ed8e1b7244e7b28fed1b93bec69c
I20261118 14:58:16.283016 1055 tablet_service.cc:1547] Processing DeleteTablet for tablet a00587922d884ced8250ac130789da68 with delete_type TABLET_DATA_DELETED (replaced by tablet 522efb108dd140b8a3fd476d202a073e) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.286904 1826 tablet_replica.cc:333] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.287261 1826 raft_consensus.cc:2261] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.288853 1055 tablet_service.cc:1547] Processing DeleteTablet for tablet 75581812cbcb48e5a2ea9d0fb925c03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.289301 1054 tablet_service.cc:1547] Processing DeleteTablet for tablet 212e4a39f2d0465a854b3474dc020aa6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5335a58da46b48019aec4b66e745fe68) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.289523 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 1b82ed8e1b7244e7b28fed1b93bec69c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.290200 1056 tablet_service.cc:1500] Processing CreateTablet for tablet 522efb108dd140b8a3fd476d202a073e (DEFAULT_TABLE table=default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.293520 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522efb108dd140b8a3fd476d202a073e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.290926 1053 tablet_service.cc:1500] Processing CreateTablet for tablet 5335a58da46b48019aec4b66e745fe68 (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.298298 1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5335a58da46b48019aec4b66e745fe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.299094 1826 raft_consensus.cc:2290] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.299693 1826 ts_tablet_manager.cc:1962] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.300076 594 catalog_manager.cc:2225] deleted orphaned HMS catalog entry for table [id=7254e5f1dc9a48a2a6fa1f0ed42fd295]
I20261118 14:58:16.300475 594 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 11) took 2477 ms. Trace:
I20261118 14:58:16.300595 594 rpcz_store.cc:276] 1118 14:58:13.822910 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.823212 (+ 302us) service_pool.cc:224] Handling call
1118 14:58:14.142825 (+319613us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:14.143361 (+ 536us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:16.006123 (+1862762us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:16.007011 (+ 888us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.007013 (+ 2us) write_op.cc:454] Released schema lock
1118 14:58:16.007296 (+ 283us) catalog_manager.cc:2222] Rolling back HMS table creation
1118 14:58:16.300440 (+293144us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:16.006879 (+ 0us) op_driver.cc:391] HandleFailure(Illegal state: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER)
1118 14:58:16.006892 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.006897 (+ 5us) write_op.cc:454] Released schema lock
1118 14:58:16.006901 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1954552,"Hive Metastore.run_cpu_time_us":489,"Hive Metastore.run_wall_time_us":200461,"spinlock_wait_cycles":5376,"child_traces":[["op",{}]]}
I20261118 14:58:16.301659 916 tablet_service.cc:1547] Processing DeleteTablet for tablet 75581812cbcb48e5a2ea9d0fb925c03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.302767 917 tablet_service.cc:1547] Processing DeleteTablet for tablet a00587922d884ced8250ac130789da68 with delete_type TABLET_DATA_DELETED (replaced by tablet 522efb108dd140b8a3fd476d202a073e) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.304083 919 tablet_service.cc:1547] Processing DeleteTablet for tablet 212e4a39f2d0465a854b3474dc020aa6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5335a58da46b48019aec4b66e745fe68) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.304383 918 tablet_service.cc:1547] Processing DeleteTablet for tablet 1b82ed8e1b7244e7b28fed1b93bec69c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.305132 915 tablet_service.cc:1500] Processing CreateTablet for tablet 522efb108dd140b8a3fd476d202a073e (DEFAULT_TABLE table=default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.305585 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522efb108dd140b8a3fd476d202a073e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.305917 1828 tablet_replica.cc:333] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.306147 917 tablet_service.cc:1500] Processing CreateTablet for tablet 5335a58da46b48019aec4b66e745fe68 (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.306545 917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5335a58da46b48019aec4b66e745fe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.306149 1828 raft_consensus.cc:2261] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.313987 1826 ts_tablet_manager.cc:1975] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.314165 1828 raft_consensus.cc:2290] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.314194 1826 log.cc:1199] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/a00587922d884ced8250ac130789da68
I20261118 14:58:16.314730 1826 ts_tablet_manager.cc:1996] T a00587922d884ced8250ac130789da68 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.314754 1828 ts_tablet_manager.cc:1962] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.320374 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:16.320829 1826 tablet_replica.cc:333] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.321092 1826 raft_consensus.cc:2261] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.321305 1826 raft_consensus.cc:2290] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.321894 1826 ts_tablet_manager.cc:1962] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.322096 1531 catalog_manager.cc:2813] Deleting table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7] in response to Hive Metastore notification log event 27
I20261118 14:58:16.324609 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet 75581812cbcb48e5a2ea9d0fb925c03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.326776 1831 tablet_replica.cc:333] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.326939 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet 212e4a39f2d0465a854b3474dc020aa6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5335a58da46b48019aec4b66e745fe68) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.327057 1831 raft_consensus.cc:2261] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.327420 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet a00587922d884ced8250ac130789da68 with delete_type TABLET_DATA_DELETED (replaced by tablet 522efb108dd140b8a3fd476d202a073e) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.327867 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet 1b82ed8e1b7244e7b28fed1b93bec69c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.328466 1190 tablet_service.cc:1500] Processing CreateTablet for tablet 522efb108dd140b8a3fd476d202a073e (DEFAULT_TABLE table=default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.329015 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522efb108dd140b8a3fd476d202a073e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.334429 1826 ts_tablet_manager.cc:1975] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.334580 1826 log.cc:1199] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/212e4a39f2d0465a854b3474dc020aa6
I20261118 14:58:16.334945 1826 ts_tablet_manager.cc:1996] T 212e4a39f2d0465a854b3474dc020aa6 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.335908 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 212e4a39f2d0465a854b3474dc020aa6 (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]) successfully deleted
I20261118 14:58:16.327441 1831 raft_consensus.cc:2290] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.336635 1189 tablet_service.cc:1500] Processing CreateTablet for tablet 5335a58da46b48019aec4b66e745fe68 (DEFAULT_TABLE table=default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.337049 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5335a58da46b48019aec4b66e745fe68. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.345784 1826 tablet_replica.cc:333] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.345988 1826 raft_consensus.cc:2261] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.346304 1826 raft_consensus.cc:2290] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.346896 1826 ts_tablet_manager.cc:1962] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.351650 1829 tablet_bootstrap.cc:492] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:16.353013 1826 ts_tablet_manager.cc:1975] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.353132 1826 log.cc:1199] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/1b82ed8e1b7244e7b28fed1b93bec69c
I20261118 14:58:16.354086 1835 tablet_bootstrap.cc:492] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:16.354102 1829 tablet_bootstrap.cc:654] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.355684 1831 ts_tablet_manager.cc:1962] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.356215 1826 ts_tablet_manager.cc:1996] T 1b82ed8e1b7244e7b28fed1b93bec69c P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.357161 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 1b82ed8e1b7244e7b28fed1b93bec69c (table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]) successfully deleted
I20261118 14:58:16.355909 1835 tablet_bootstrap.cc:654] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.364234 1826 tablet_replica.cc:333] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.364574 1826 raft_consensus.cc:2261] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.364805 1826 raft_consensus.cc:2290] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.365087 1531 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet 5335a58da46b48019aec4b66e745fe68
I20261118 14:58:16.365518 1826 ts_tablet_manager.cc:1962] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.365870 915 tablet_service.cc:1547] Processing DeleteTablet for tablet 5335a58da46b48019aec4b66e745fe68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.366921 1827 tablet_bootstrap.cc:492] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:16.368932 1827 tablet_bootstrap.cc:654] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.371999 1831 ts_tablet_manager.cc:1975] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.372972 1842 tablet.cc:1759] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20261118 14:58:16.374684 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 5335a58da46b48019aec4b66e745fe68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.378401 1841 tablet.cc:1759] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20261118 14:58:16.383252 1826 ts_tablet_manager.cc:1975] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.383415 1826 log.cc:1199] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/75581812cbcb48e5a2ea9d0fb925c03a
I20261118 14:58:16.383808 1826 ts_tablet_manager.cc:1996] T 75581812cbcb48e5a2ea9d0fb925c03a P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.384903 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 75581812cbcb48e5a2ea9d0fb925c03a (table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]) successfully deleted
W20261118 14:58:16.385491 1466 catalog_manager.cc:5045] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): delete failed for tablet 5335a58da46b48019aec4b66e745fe68 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5335a58da46b48019aec4b66e745fe68 already in progress: creating tablet
I20261118 14:58:16.386277 1828 ts_tablet_manager.cc:1975] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.386556 1124 heartbeater.cc:500] Master 127.0.72.189:46707 was elected leader, sending a full tablet report...
I20261118 14:58:16.388556 1828 log.cc:1199] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/75581812cbcb48e5a2ea9d0fb925c03a
I20261118 14:58:16.389029 1828 ts_tablet_manager.cc:1996] T 75581812cbcb48e5a2ea9d0fb925c03a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.391038 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 75581812cbcb48e5a2ea9d0fb925c03a (table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]) successfully deleted
I20261118 14:58:16.391510 1829 tablet_bootstrap.cc:492] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:16.391666 1829 ts_tablet_manager.cc:1449] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20261118 14:58:16.392503 1829 raft_consensus.cc:360] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.392704 1829 raft_consensus.cc:386] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.392788 1829 raft_consensus.cc:748] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.394312 1827 tablet_bootstrap.cc:492] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:16.394563 1827 ts_tablet_manager.cc:1449] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.001s
I20261118 14:58:16.372808 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet 5335a58da46b48019aec4b66e745fe68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.395126 1831 log.cc:1199] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/75581812cbcb48e5a2ea9d0fb925c03a
I20261118 14:58:16.395324 1827 raft_consensus.cc:360] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.395654 1831 ts_tablet_manager.cc:1996] T 75581812cbcb48e5a2ea9d0fb925c03a P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.396317 1827 raft_consensus.cc:386] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.396932 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 75581812cbcb48e5a2ea9d0fb925c03a (table default.table_6c4a501fa569491080a3b4988350b5b3 [id=91e39a36391d48d99bab0efaef13cdfe]) successfully deleted
I20261118 14:58:16.398018 1827 raft_consensus.cc:748] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.398558 1831 tablet_replica.cc:333] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.398569 1827 consensus_queue.cc:261] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.398828 1831 raft_consensus.cc:2261] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.399072 1831 raft_consensus.cc:2290] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.399698 1831 ts_tablet_manager.cc:1962] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.401136 1827 ts_tablet_manager.cc:1480] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20261118 14:58:16.401995 1827 tablet_bootstrap.cc:492] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:16.404053 1827 tablet_bootstrap.cc:654] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.406709 1831 ts_tablet_manager.cc:1975] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.406888 1831 log.cc:1199] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/212e4a39f2d0465a854b3474dc020aa6
I20261118 14:58:16.407321 1831 ts_tablet_manager.cc:1996] T 212e4a39f2d0465a854b3474dc020aa6 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.407984 1828 tablet_replica.cc:333] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.408273 1828 raft_consensus.cc:2261] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.397730 1844 tablet.cc:1759] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20261118 14:58:16.408995 1831 tablet_replica.cc:333] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.409245 1831 raft_consensus.cc:2261] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.409464 1831 raft_consensus.cc:2290] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.409587 1828 raft_consensus.cc:2290] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.410138 1831 ts_tablet_manager.cc:1962] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.409790 1829 consensus_queue.cc:261] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.411046 1835 tablet_bootstrap.cc:492] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:16.411206 1835 ts_tablet_manager.cc:1449] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.008s
I20261118 14:58:16.412161 1835 raft_consensus.cc:360] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.412410 1835 raft_consensus.cc:386] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.412596 1835 raft_consensus.cc:748] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.412621 1828 ts_tablet_manager.cc:1962] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.412943 1835 consensus_queue.cc:261] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.413789 1835 ts_tablet_manager.cc:1480] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:16.414116 1835 tablet_bootstrap.cc:492] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:16.419148 1259 heartbeater.cc:500] Master 127.0.72.189:46707 was elected leader, sending a full tablet report...
W20261118 14:58:16.431195 874 consensus_peers.cc:597] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f -> Peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Couldn't send request to peer c01f1b88968f4905ba5a401a81ade9f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 212e4a39f2d0465a854b3474dc020aa6. This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:16.431599 598 catalog_manager.cc:2225] deleted orphaned HMS catalog entry for table [id=8cc4e4385a0b474baca66e94b8a7346d]
I20261118 14:58:16.431936 598 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 16) took 2600 ms. Trace:
I20261118 14:58:16.432032 598 rpcz_store.cc:276] 1118 14:58:13.831623 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.884891 (+ 53268us) service_pool.cc:224] Handling call
1118 14:58:13.890748 (+ 5857us) spinlock_profiling.cc:245] Waited 5.61 ms on lock 0x616000043354. stack: 00007f26cab673a6 00007f26cab67160 00007f26c9b5834d 00007f26e112b704 00007f26e112b6cd 00007f26da07d82d 00007f26da07d923 00007f26d6be7221 00007f26deadc50d 00007f26dec6b873 00007f26d5574526 00007f26d5574255 00007f26d1caf8de 00007f26d1cae7be 00007f26d1cb49f8 00007f26d1cb535f
1118 14:58:14.143402 (+252654us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:14.143870 (+ 468us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:16.028800 (+1884930us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:16.029697 (+ 897us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.029701 (+ 4us) write_op.cc:454] Released schema lock
1118 14:58:16.031029 (+ 1328us) catalog_manager.cc:2222] Rolling back HMS table creation
1118 14:58:16.431877 (+400848us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:16.029611 (+ 0us) op_driver.cc:391] HandleFailure(Illegal state: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER)
1118 14:58:16.029625 (+ 14us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.029632 (+ 7us) write_op.cc:454] Released schema lock
1118 14:58:16.029637 (+ 5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2130919,"Hive Metastore.run_cpu_time_us":476,"Hive Metastore.run_wall_time_us":153904,"spinlock_wait_cycles":12901504,"child_traces":[["op",{}]]}
I20261118 14:58:16.433065 1835 tablet_bootstrap.cc:654] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.444059 1829 ts_tablet_manager.cc:1480] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.052s user 0.002s sys 0.000s
I20261118 14:58:16.451573 1835 tablet_bootstrap.cc:492] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:16.451767 1835 ts_tablet_manager.cc:1449] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.000s
I20261118 14:58:16.452610 1835 raft_consensus.cc:360] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.452818 1835 raft_consensus.cc:386] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.452903 1835 raft_consensus.cc:748] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.453157 1835 consensus_queue.cc:261] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.453860 1835 ts_tablet_manager.cc:1480] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:16.463368 986 heartbeater.cc:500] Master 127.0.72.189:46707 was elected leader, sending a full tablet report...
I20261118 14:58:16.465562 1829 tablet_bootstrap.cc:492] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:16.467608 1829 tablet_bootstrap.cc:654] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.470041 1831 ts_tablet_manager.cc:1975] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.477545 1831 log.cc:1199] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/a00587922d884ced8250ac130789da68
I20261118 14:58:16.478103 1831 ts_tablet_manager.cc:1996] T a00587922d884ced8250ac130789da68 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.479440 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:16.480101 1831 tablet_replica.cc:333] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.480326 1831 raft_consensus.cc:2261] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.480575 1831 raft_consensus.cc:2290] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.480924 1827 tablet_bootstrap.cc:492] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:16.481142 1827 ts_tablet_manager.cc:1449] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.079s user 0.036s sys 0.028s
I20261118 14:58:16.481243 1831 ts_tablet_manager.cc:1962] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.481946 1827 raft_consensus.cc:360] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.482187 1827 raft_consensus.cc:386] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.482286 1827 raft_consensus.cc:748] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.482574 1827 consensus_queue.cc:261] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.483120 1827 ts_tablet_manager.cc:1480] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20261118 14:58:16.484437 1829 tablet_bootstrap.cc:492] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:16.486263 1829 ts_tablet_manager.cc:1449] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20261118 14:58:16.476403 1828 ts_tablet_manager.cc:1975] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.488050 1828 log.cc:1199] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/a00587922d884ced8250ac130789da68
I20261118 14:58:16.488503 1828 ts_tablet_manager.cc:1996] T a00587922d884ced8250ac130789da68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.488530 1829 raft_consensus.cc:360] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.488790 1829 raft_consensus.cc:386] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.488910 1829 raft_consensus.cc:748] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.489590 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet a00587922d884ced8250ac130789da68 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:16.489959 1828 tablet_replica.cc:333] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.490095 1829 consensus_queue.cc:261] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.492266 1829 ts_tablet_manager.cc:1480] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20261118 14:58:16.490185 1828 raft_consensus.cc:2261] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.492792 1828 raft_consensus.cc:2290] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.495536 1828 ts_tablet_manager.cc:1962] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.503427 1828 ts_tablet_manager.cc:1975] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.503597 1828 log.cc:1199] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/212e4a39f2d0465a854b3474dc020aa6
I20261118 14:58:16.504006 1828 ts_tablet_manager.cc:1996] T 212e4a39f2d0465a854b3474dc020aa6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.504457 1831 ts_tablet_manager.cc:1975] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.504629 1831 log.cc:1199] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/1b82ed8e1b7244e7b28fed1b93bec69c
I20261118 14:58:16.505061 1831 ts_tablet_manager.cc:1996] T 1b82ed8e1b7244e7b28fed1b93bec69c P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.505288 1828 tablet_replica.cc:333] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.505529 1828 raft_consensus.cc:2261] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.505728 1828 raft_consensus.cc:2290] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.506115 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 1b82ed8e1b7244e7b28fed1b93bec69c (table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]) successfully deleted
I20261118 14:58:16.506342 1828 ts_tablet_manager.cc:1962] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.506500 1831 tablet_replica.cc:333] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.506719 1831 raft_consensus.cc:2261] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.506935 1831 raft_consensus.cc:2290] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.507637 1831 ts_tablet_manager.cc:1962] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.511649 595 catalog_manager.cc:2225] deleted orphaned HMS catalog entry for table [id=c32b621587d1449d8931a348b7f8b654]
I20261118 14:58:16.511955 595 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 10) took 2689 ms. Trace:
I20261118 14:58:16.512254 1828 ts_tablet_manager.cc:1975] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.512081 595 rpcz_store.cc:276] 1118 14:58:13.822812 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.823787 (+ 975us) service_pool.cc:224] Handling call
1118 14:58:14.143897 (+320110us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:14.144319 (+ 422us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:16.051031 (+1906712us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:16.051803 (+ 772us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.051805 (+ 2us) write_op.cc:454] Released schema lock
1118 14:58:16.052003 (+ 198us) catalog_manager.cc:2222] Rolling back HMS table creation
1118 14:58:16.511919 (+459916us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:16.051743 (+ 0us) op_driver.cc:391] HandleFailure(Illegal state: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER)
1118 14:58:16.051753 (+ 10us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.051758 (+ 5us) write_op.cc:454] Released schema lock
1118 14:58:16.051761 (+ 3us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2263451,"Hive Metastore.run_cpu_time_us":529,"Hive Metastore.run_wall_time_us":102483,"child_traces":[["op",{}]]}
I20261118 14:58:16.512535 1828 log.cc:1199] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/1b82ed8e1b7244e7b28fed1b93bec69c
I20261118 14:58:16.512914 1828 ts_tablet_manager.cc:1996] T 1b82ed8e1b7244e7b28fed1b93bec69c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.513875 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 1b82ed8e1b7244e7b28fed1b93bec69c (table default.table_41739c72be484e8fb242c925dd0e2c32 [id=1b827df58389480588f081062466b985]) successfully deleted
I20261118 14:58:16.514062 1831 ts_tablet_manager.cc:1975] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:16.514226 1831 log.cc:1199] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/5335a58da46b48019aec4b66e745fe68
I20261118 14:58:16.514237 1828 tablet_replica.cc:333] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.514412 1828 raft_consensus.cc:2261] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.514629 1831 ts_tablet_manager.cc:1996] T 5335a58da46b48019aec4b66e745fe68 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.514627 1828 raft_consensus.cc:2290] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.515225 1828 ts_tablet_manager.cc:1962] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.515617 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 5335a58da46b48019aec4b66e745fe68 (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]) successfully deleted
I20261118 14:58:16.523154 1828 ts_tablet_manager.cc:1975] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:16.523345 1828 log.cc:1199] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/5335a58da46b48019aec4b66e745fe68
I20261118 14:58:16.523756 1828 ts_tablet_manager.cc:1996] T 5335a58da46b48019aec4b66e745fe68 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.524753 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 5335a58da46b48019aec4b66e745fe68 (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]) successfully deleted
I20261118 14:58:16.527865 1479 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]
I20261118 14:58:16.531903 1531 catalog_manager.cc:2813] Deleting table default.table_4dba69207cca45529e828162200c13f5 [id=7254e5f1dc9a48a2a6fa1f0ed42fd295] in response to Hive Metastore notification log event 30
W20261118 14:58:16.532235 1531 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 30 DROP_TABLE default.table_4dba69207cca45529e828162200c13f5: Not found: the table does not exist: table_id: "7254e5f1dc9a48a2a6fa1f0ed42fd295" table_name: "default.table_4dba69207cca45529e828162200c13f5"
I20261118 14:58:16.533099 1531 catalog_manager.cc:2813] Deleting table default.table_053874e1edfe4b8eac52b4ba5f1b1870 [id=8cc4e4385a0b474baca66e94b8a7346d] in response to Hive Metastore notification log event 31
W20261118 14:58:16.533310 1531 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 31 DROP_TABLE default.table_053874e1edfe4b8eac52b4ba5f1b1870: Not found: the table does not exist: table_id: "8cc4e4385a0b474baca66e94b8a7346d" table_name: "default.table_053874e1edfe4b8eac52b4ba5f1b1870"
I20261118 14:58:16.534073 1531 catalog_manager.cc:2813] Deleting table default.table_d24daba84f8c4c94824da319564f0453 [id=c32b621587d1449d8931a348b7f8b654] in response to Hive Metastore notification log event 32
W20261118 14:58:16.534261 1531 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 32 DROP_TABLE default.table_d24daba84f8c4c94824da319564f0453: Not found: the table does not exist: table_id: "c32b621587d1449d8931a348b7f8b654" table_name: "default.table_d24daba84f8c4c94824da319564f0453"
I20261118 14:58:16.535239 1531 catalog_manager.cc:2813] Deleting table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f] in response to Hive Metastore notification log event 33
I20261118 14:58:16.547298 1531 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet 8a07aac51e8a48dba2c61e9e477fad11
W20261118 14:58:16.549013 1478 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_7171b2f0b6ea470393bf9aa28246cf65 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:16.551513 1473 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:16.551790 1473 catalog_manager.cc:5840] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "c01f1b88968f4905ba5a401a81ade9f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } }
W20261118 14:58:16.552560 1477 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 14:58:16.552248 1470 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_155ca12a699d4393b30979580a04864a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:16.555166 1475 master_service.cc:965] ReplaceTablet: received request to replace tablet cf7800d54792490faa3860de0ecb7a3a from {username='slave'} at 127.0.0.1:50956
W20261118 14:58:16.555619 1472 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_9021a49aff8546eca95c57bde20822b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 14:58:16.557720 915 tablet_service.cc:1547] Processing DeleteTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.558058 1828 tablet_replica.cc:333] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.558266 1828 raft_consensus.cc:2261] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.558471 1828 raft_consensus.cc:2290] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.559132 1828 ts_tablet_manager.cc:1962] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.559296 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.559585 1826 tablet_replica.cc:333] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.559751 1826 raft_consensus.cc:2261] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.560065 1826 raft_consensus.cc:2290] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.560608 1826 ts_tablet_manager.cc:1962] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.563155 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.563946 1831 tablet_replica.cc:333] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.564139 1831 raft_consensus.cc:2261] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.564314 1831 raft_consensus.cc:2290] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.564934 1831 ts_tablet_manager.cc:1962] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.566985 1826 ts_tablet_manager.cc:1975] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.567091 1826 log.cc:1199] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/8a07aac51e8a48dba2c61e9e477fad11
I20261118 14:58:16.567497 1826 ts_tablet_manager.cc:1996] T 8a07aac51e8a48dba2c61e9e477fad11 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.568521 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 8a07aac51e8a48dba2c61e9e477fad11 (table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]) successfully deleted
I20261118 14:58:16.578468 1828 ts_tablet_manager.cc:1975] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.582101 1831 ts_tablet_manager.cc:1975] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:16.582276 1831 log.cc:1199] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/8a07aac51e8a48dba2c61e9e477fad11
I20261118 14:58:16.582717 1831 ts_tablet_manager.cc:1996] T 8a07aac51e8a48dba2c61e9e477fad11 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.582791 1475 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet cf7800d54792490faa3860de0ecb7a3a
I20261118 14:58:16.583093 1828 log.cc:1199] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/8a07aac51e8a48dba2c61e9e477fad11
I20261118 14:58:16.583495 1828 ts_tablet_manager.cc:1996] T 8a07aac51e8a48dba2c61e9e477fad11 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.583938 1475 catalog_manager.cc:6936] ReplaceTablet: tablet cf7800d54792490faa3860de0ecb7a3a deleted and replaced by tablet c8354c983bc5487e9c94202616d222c6
I20261118 14:58:16.584192 915 tablet_service.cc:1547] Processing DeleteTablet for tablet cf7800d54792490faa3860de0ecb7a3a with delete_type TABLET_DATA_DELETED (replaced by tablet c8354c983bc5487e9c94202616d222c6) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.584331 1475 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_0cbce4af337d4a5baeb313a32662d446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:16.584494 1828 tablet_replica.cc:333] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.584681 1828 raft_consensus.cc:2261] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.584801 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 8a07aac51e8a48dba2c61e9e477fad11 (table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]) successfully deleted
I20261118 14:58:16.585037 1828 raft_consensus.cc:2290] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.585337 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet cf7800d54792490faa3860de0ecb7a3a with delete_type TABLET_DATA_DELETED (replaced by tablet c8354c983bc5487e9c94202616d222c6) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.585600 1826 tablet_replica.cc:333] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.585608 1828 ts_tablet_manager.cc:1962] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.585793 1826 raft_consensus.cc:2261] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.585981 1826 raft_consensus.cc:2290] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.586632 1826 ts_tablet_manager.cc:1962] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.589638 597 catalog_manager.cc:2225] deleted orphaned HMS catalog entry for table [id=0ce3ba58cd7440a9b27427e7713af331]
I20261118 14:58:16.589938 597 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 14) took 2766 ms. Trace:
I20261118 14:58:16.590027 597 rpcz_store.cc:276] 1118 14:58:13.823334 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.848696 (+ 25362us) service_pool.cc:224] Handling call
1118 14:58:13.883053 (+ 34357us) spinlock_profiling.cc:245] Waited 34 ms on lock 0x616000043354. stack: 00007f26cab673a6 00007f26cab67160 00007f26c9b5834d 00007f26e112b704 00007f26e112b6cd 00007f26da07d82d 00007f26da07d923 00007f26d6be7221 00007f26deadc50d 00007f26dec6b873 00007f26d5574526 00007f26d5574255 00007f26d1caf8de 00007f26d1cae7be 00007f26d1cb49f8 00007f26d1cb535f
1118 14:58:14.144220 (+261167us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:14.144640 (+ 420us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:16.105429 (+1960789us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:16.106147 (+ 718us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.106149 (+ 2us) write_op.cc:454] Released schema lock
1118 14:58:16.106337 (+ 188us) catalog_manager.cc:2222] Rolling back HMS table creation
1118 14:58:16.589880 (+483543us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:16.106087 (+ 0us) op_driver.cc:391] HandleFailure(Illegal state: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER)
1118 14:58:16.106097 (+ 10us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.106103 (+ 6us) write_op.cc:454] Released schema lock
1118 14:58:16.106107 (+ 4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2346351,"Hive Metastore.run_cpu_time_us":515,"Hive Metastore.run_wall_time_us":97133,"spinlock_wait_cycles":78217984,"child_traces":[["op",{}]]}
I20261118 14:58:16.591624 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet cf7800d54792490faa3860de0ecb7a3a with delete_type TABLET_DATA_DELETED (replaced by tablet c8354c983bc5487e9c94202616d222c6) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.593660 1474 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:16.599498 1831 tablet_replica.cc:333] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.599686 1826 ts_tablet_manager.cc:1975] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.599702 1831 raft_consensus.cc:2261] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.599725 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 8a07aac51e8a48dba2c61e9e477fad11 (table default.table_ced554c65cd4465d9ddab052dc32efcf [id=e4b541d82e0f4d0991446b865e206f8f]) successfully deleted
I20261118 14:58:16.599905 1831 raft_consensus.cc:2290] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.599926 1826 log.cc:1199] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/cf7800d54792490faa3860de0ecb7a3a
I20261118 14:58:16.600402 1826 ts_tablet_manager.cc:1996] T cf7800d54792490faa3860de0ecb7a3a P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.600513 1831 ts_tablet_manager.cc:1962] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.601635 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet cf7800d54792490faa3860de0ecb7a3a (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]) successfully deleted
I20261118 14:58:16.606532 915 tablet_service.cc:1500] Processing CreateTablet for tablet c8354c983bc5487e9c94202616d222c6 (DEFAULT_TABLE table=default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.606987 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8354c983bc5487e9c94202616d222c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.607057 1476 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:16.610226 1056 tablet_service.cc:1500] Processing CreateTablet for tablet c8354c983bc5487e9c94202616d222c6 (DEFAULT_TABLE table=default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.610670 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8354c983bc5487e9c94202616d222c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.610853 1190 tablet_service.cc:1500] Processing CreateTablet for tablet c8354c983bc5487e9c94202616d222c6 (DEFAULT_TABLE table=default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.611830 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8354c983bc5487e9c94202616d222c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.617208 1827 tablet_bootstrap.cc:492] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:16.618940 1827 tablet_bootstrap.cc:654] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.622505 1473 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d] alter complete (version 2)
I20261118 14:58:16.624118 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 5335a58da46b48019aec4b66e745fe68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.624486 1826 tablet_replica.cc:333] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.625469 1826 raft_consensus.cc:2261] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Raft consensus shutting down.
W20261118 14:58:16.626197 1476 catalog_manager.cc:6040] Failed to send AlterTable RPC for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 522efb108dd140b8a3fd476d202a073e consensus state has no leader
I20261118 14:58:16.626823 916 tablet_service.cc:1547] Processing DeleteTablet for tablet 5335a58da46b48019aec4b66e745fe68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.626959 1476 master_service.cc:965] ReplaceTablet: received request to replace tablet a00587922d884ced8250ac130789da68 from {username='slave'} at 127.0.0.1:50956
I20261118 14:58:16.627033 918 tablet_service.cc:1547] Processing DeleteTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.627269 919 tablet_service.cc:1547] Processing DeleteTablet for tablet a00587922d884ced8250ac130789da68 with delete_type TABLET_DATA_DELETED (replaced by tablet 522efb108dd140b8a3fd476d202a073e) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.627405 917 tablet_service.cc:1547] Processing DeleteTablet for tablet 1b82ed8e1b7244e7b28fed1b93bec69c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.627846 1476 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" } modify_external_catalogs: true
I20261118 14:58:16.628062 1476 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" } modify_external_catalogs: true
I20261118 14:58:16.628509 1826 raft_consensus.cc:2290] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.624691 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.629887 1826 ts_tablet_manager.cc:1962] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.631448 1828 ts_tablet_manager.cc:1975] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.631587 1828 log.cc:1199] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/cf7800d54792490faa3860de0ecb7a3a
I20261118 14:58:16.631754 917 tablet_service.cc:1547] Processing DeleteTablet for tablet 212e4a39f2d0465a854b3474dc020aa6 with delete_type TABLET_DATA_DELETED (replaced by tablet 5335a58da46b48019aec4b66e745fe68) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.632033 1828 ts_tablet_manager.cc:1996] T cf7800d54792490faa3860de0ecb7a3a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.633404 1831 ts_tablet_manager.cc:1975] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.633545 1831 log.cc:1199] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/cf7800d54792490faa3860de0ecb7a3a
I20261118 14:58:16.634020 1831 ts_tablet_manager.cc:1996] T cf7800d54792490faa3860de0ecb7a3a P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.635146 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet cf7800d54792490faa3860de0ecb7a3a (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]) successfully deleted
I20261118 14:58:16.636034 1835 tablet_bootstrap.cc:492] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:16.637482 1474 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:16.637754 1835 tablet_bootstrap.cc:654] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.633060 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet cf7800d54792490faa3860de0ecb7a3a (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]) successfully deleted
W20261118 14:58:16.638886 1463 catalog_manager.cc:5032] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet 5335a58da46b48019aec4b66e745fe68 because the tablet was not found. No further retry: Not found: Tablet not found: 5335a58da46b48019aec4b66e745fe68
W20261118 14:58:16.639086 1463 catalog_manager.cc:5032] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet 8a07aac51e8a48dba2c61e9e477fad11 because the tablet was not found. No further retry: Not found: Tablet not found: 8a07aac51e8a48dba2c61e9e477fad11
W20261118 14:58:16.639271 1463 catalog_manager.cc:5032] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet a00587922d884ced8250ac130789da68 because the tablet was not found. No further retry: Not found: Tablet not found: a00587922d884ced8250ac130789da68
W20261118 14:58:16.639428 1463 catalog_manager.cc:5032] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet 1b82ed8e1b7244e7b28fed1b93bec69c because the tablet was not found. No further retry: Not found: Tablet not found: 1b82ed8e1b7244e7b28fed1b93bec69c
W20261118 14:58:16.639787 1463 catalog_manager.cc:5032] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): delete failed for tablet 212e4a39f2d0465a854b3474dc020aa6 because the tablet was not found. No further retry: Not found: Tablet not found: 212e4a39f2d0465a854b3474dc020aa6
I20261118 14:58:16.640504 1473 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1" } modify_external_catalogs: true
I20261118 14:58:16.640712 1473 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1" } modify_external_catalogs: true
I20261118 14:58:16.641098 1829 tablet_bootstrap.cc:492] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:16.641651 1191 tablet_service.cc:1547] Processing DeleteTablet for tablet 8a07aac51e8a48dba2c61e9e477fad11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.641927 1187 tablet_service.cc:1547] Processing DeleteTablet for tablet cf7800d54792490faa3860de0ecb7a3a with delete_type TABLET_DATA_DELETED (replaced by tablet c8354c983bc5487e9c94202616d222c6) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.642545 1192 tablet_service.cc:1547] Processing DeleteTablet for tablet 1b82ed8e1b7244e7b28fed1b93bec69c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.642633 1189 tablet_service.cc:1547] Processing DeleteTablet for tablet 5335a58da46b48019aec4b66e745fe68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 14:58:16 UTC) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.642824 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet a00587922d884ced8250ac130789da68 with delete_type TABLET_DATA_DELETED (replaced by tablet 522efb108dd140b8a3fd476d202a073e) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.643394 1829 tablet_bootstrap.cc:654] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
W20261118 14:58:16.643705 1465 catalog_manager.cc:5032] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): delete failed for tablet 8a07aac51e8a48dba2c61e9e477fad11 because the tablet was not found. No further retry: Not found: Tablet not found: 8a07aac51e8a48dba2c61e9e477fad11
W20261118 14:58:16.643905 1465 catalog_manager.cc:5032] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): delete failed for tablet cf7800d54792490faa3860de0ecb7a3a because the tablet was not found. No further retry: Not found: Tablet not found: cf7800d54792490faa3860de0ecb7a3a
W20261118 14:58:16.644105 1465 catalog_manager.cc:5032] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): delete failed for tablet 1b82ed8e1b7244e7b28fed1b93bec69c because the tablet was not found. No further retry: Not found: Tablet not found: 1b82ed8e1b7244e7b28fed1b93bec69c
W20261118 14:58:16.644304 1465 catalog_manager.cc:5032] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): delete failed for tablet 5335a58da46b48019aec4b66e745fe68 because the tablet was not found. No further retry: Not found: Tablet not found: 5335a58da46b48019aec4b66e745fe68
W20261118 14:58:16.644562 1465 catalog_manager.cc:5032] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): delete failed for tablet a00587922d884ced8250ac130789da68 because the tablet was not found. No further retry: Not found: Tablet not found: a00587922d884ced8250ac130789da68
I20261118 14:58:16.647560 1478 catalog_manager.cc:2215] created HMS catalog entry for table [id=cdc2cda186804cd39b0d320f08e91df8]
I20261118 14:58:16.655815 1478 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:16.657083 1478 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9" } modify_external_catalogs: true
I20261118 14:58:16.657373 1478 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9" } modify_external_catalogs: true
I20261118 14:58:16.669044 1829 tablet_bootstrap.cc:492] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:16.669227 1829 ts_tablet_manager.cc:1449] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.006s
I20261118 14:58:16.669984 1829 raft_consensus.cc:360] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.670217 1829 raft_consensus.cc:386] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.670311 1829 raft_consensus.cc:748] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.670568 1829 consensus_queue.cc:261] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.671178 1829 ts_tablet_manager.cc:1480] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20261118 14:58:16.673322 1835 tablet_bootstrap.cc:492] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:16.673513 1835 ts_tablet_manager.cc:1449] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.000s
I20261118 14:58:16.674294 1835 raft_consensus.cc:360] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.674556 1835 raft_consensus.cc:386] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.674695 1835 raft_consensus.cc:748] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.675005 1835 consensus_queue.cc:261] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.675767 1056 tablet_service.cc:1500] Processing CreateTablet for tablet cafee7e386434b3c85542d7b6c5024ed (DEFAULT_TABLE table=default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.677000 1835 ts_tablet_manager.cc:1480] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20261118 14:58:16.686533 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cafee7e386434b3c85542d7b6c5024ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.686755 602 catalog_manager.cc:2225] deleted orphaned HMS catalog entry for table [id=378bd168f216477992c1b7bb33791778]
I20261118 14:58:16.687088 602 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:60882 (request call id 4) took 2864 ms. Trace:
I20261118 14:58:16.687191 602 rpcz_store.cc:276] 1118 14:58:13.822144 (+ 0us) service_pool.cc:167] Inserting onto call queue
1118 14:58:13.826668 (+ 4524us) service_pool.cc:224] Handling call
1118 14:58:14.144890 (+318222us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 14:58:14.145312 (+ 422us) catalog_manager.cc:2188] Created new table and tablet info
1118 14:58:16.126845 (+1981533us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 14:58:16.127739 (+ 894us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.127743 (+ 4us) write_op.cc:454] Released schema lock
1118 14:58:16.128004 (+ 261us) catalog_manager.cc:2222] Rolling back HMS table creation
1118 14:58:16.687027 (+559023us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 14:58:16.127655 (+ 0us) op_driver.cc:391] HandleFailure(Illegal state: replica 0ed65a3719634e519d783540afd6189e is not leader of this config: current role FOLLOWER)
1118 14:58:16.127669 (+ 14us) write_op.cc:489] Releasing partition, row and schema locks
1118 14:58:16.127677 (+ 8us) write_op.cc:454] Released schema lock
1118 14:58:16.127683 (+ 6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2421274,"Hive Metastore.run_cpu_time_us":515,"Hive Metastore.run_wall_time_us":118660,"spinlock_wait_cycles":6784,"child_traces":[["op",{}]]}
W20261118 14:58:16.688429 1465 catalog_manager.cc:4847] Async tablet task AlterTable RPC for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 522efb108dd140b8a3fd476d202a073e consensus state has no leader
I20261118 14:58:16.690430 915 tablet_service.cc:1500] Processing CreateTablet for tablet cafee7e386434b3c85542d7b6c5024ed (DEFAULT_TABLE table=default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.690887 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cafee7e386434b3c85542d7b6c5024ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.696026 1190 tablet_service.cc:1500] Processing CreateTablet for tablet cafee7e386434b3c85542d7b6c5024ed (DEFAULT_TABLE table=default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.696396 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cafee7e386434b3c85542d7b6c5024ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.697995 1474 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50956:
table { table_name: "default.Table_a0ea13e08f4e45bc84d5936c615c8ff7" } new_table_name: "default.Table_8afe8b6b4fe4433882ded7831ce722e9" modify_external_catalogs: true
I20261118 14:58:16.703320 1835 tablet_bootstrap.cc:492] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:16.704916 1835 tablet_bootstrap.cc:654] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.711133 1827 tablet_bootstrap.cc:492] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:16.711324 1827 ts_tablet_manager.cc:1449] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.094s user 0.008s sys 0.001s
I20261118 14:58:16.712065 1827 raft_consensus.cc:360] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.712298 1827 raft_consensus.cc:386] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.712502 1827 raft_consensus.cc:748] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.712800 1827 consensus_queue.cc:261] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.716810 1827 ts_tablet_manager.cc:1480] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20261118 14:58:16.718266 1477 catalog_manager.cc:2215] created HMS catalog entry for table [id=d65d9ba97cda44e3bdff3add1fec1ec8]
I20261118 14:58:16.720404 1829 tablet_bootstrap.cc:492] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:16.722220 1829 tablet_bootstrap.cc:654] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.730791 1477 catalog_manager.cc:5940] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463) does not have the latest schema for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]). Expected version 1 got 0
W20261118 14:58:16.731424 1477 catalog_manager.cc:6040] Failed to send AlterTable RPC for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet c8354c983bc5487e9c94202616d222c6 consensus state has no leader
I20261118 14:58:16.731774 1826 ts_tablet_manager.cc:1975] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 14:58:16.731887 1826 log.cc:1199] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/5335a58da46b48019aec4b66e745fe68
I20261118 14:58:16.732224 1826 ts_tablet_manager.cc:1996] T 5335a58da46b48019aec4b66e745fe68 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.732527 1477 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]). Expected version 1 got 0
I20261118 14:58:16.732846 1827 tablet_bootstrap.cc:492] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:16.734494 1827 tablet_bootstrap.cc:654] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.742072 1829 tablet_bootstrap.cc:492] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:16.742291 1829 ts_tablet_manager.cc:1449] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.007s
I20261118 14:58:16.743076 1829 raft_consensus.cc:360] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.754681 1056 tablet_service.cc:1500] Processing CreateTablet for tablet 96011b926cca4e4bb2f66fc7944201f7 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.755102 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96011b926cca4e4bb2f66fc7944201f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.757066 1477 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]). Expected version 1 got 0
I20261118 14:58:16.743340 1829 raft_consensus.cc:386] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.757299 1829 raft_consensus.cc:748] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.758191 1829 consensus_queue.cc:261] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.758267 1477 master_service.cc:965] ReplaceTablet: received request to replace tablet bbf3d70de6b143e88c9dfc83f7648e8f from {username='slave'} at 127.0.0.1:50956
I20261118 14:58:16.759629 1829 ts_tablet_manager.cc:1480] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.017s user 0.000s sys 0.002s
I20261118 14:58:16.752907 915 tablet_service.cc:1500] Processing CreateTablet for tablet 96011b926cca4e4bb2f66fc7944201f7 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.760092 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96011b926cca4e4bb2f66fc7944201f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.761485 1190 tablet_service.cc:1500] Processing CreateTablet for tablet 96011b926cca4e4bb2f66fc7944201f7 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.762089 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96011b926cca4e4bb2f66fc7944201f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.768608 1477 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet bbf3d70de6b143e88c9dfc83f7648e8f
I20261118 14:58:16.769759 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet bbf3d70de6b143e88c9dfc83f7648e8f with delete_type TABLET_DATA_DELETED (replaced by tablet 23b3e64e87dd4302a68a8cff81a03da1) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.771199 917 tablet_service.cc:1547] Processing DeleteTablet for tablet bbf3d70de6b143e88c9dfc83f7648e8f with delete_type TABLET_DATA_DELETED (replaced by tablet 23b3e64e87dd4302a68a8cff81a03da1) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.771459 1829 tablet_bootstrap.cc:492] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:16.772085 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet bbf3d70de6b143e88c9dfc83f7648e8f with delete_type TABLET_DATA_DELETED (replaced by tablet 23b3e64e87dd4302a68a8cff81a03da1) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.772358 1831 tablet_replica.cc:333] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.772584 1831 raft_consensus.cc:2261] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.772773 1831 raft_consensus.cc:2290] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.771523 1828 tablet_replica.cc:333] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.773897 1831 ts_tablet_manager.cc:1962] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.774530 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 5335a58da46b48019aec4b66e745fe68 (table default.table_d908b160aac24b678c5aff465735ea9f [id=8656240a083440b4b78095548a82a9c7]) successfully deleted
W20261118 14:58:16.774811 1466 catalog_manager.cc:5032] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): delete failed for tablet 8a07aac51e8a48dba2c61e9e477fad11 because the tablet was not found. No further retry: Not found: Tablet not found: 8a07aac51e8a48dba2c61e9e477fad11
I20261118 14:58:16.776850 1828 raft_consensus.cc:2261] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.777238 1828 raft_consensus.cc:2290] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.777343 1826 tablet_replica.cc:333] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.777890 1826 raft_consensus.cc:2261] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.778110 1826 raft_consensus.cc:2290] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.778380 1835 tablet_bootstrap.cc:492] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:16.778570 1835 ts_tablet_manager.cc:1449] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.075s user 0.005s sys 0.007s
I20261118 14:58:16.778779 1826 ts_tablet_manager.cc:1962] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.779374 1835 raft_consensus.cc:360] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.779587 1835 raft_consensus.cc:386] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.779666 1835 raft_consensus.cc:748] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.779925 1835 consensus_queue.cc:261] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.780535 1835 ts_tablet_manager.cc:1480] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 14:58:16.781726 1827 tablet_bootstrap.cc:492] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:16.781924 1827 ts_tablet_manager.cc:1449] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.001s
I20261118 14:58:16.782836 1827 raft_consensus.cc:360] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.783192 1477 catalog_manager.cc:6936] ReplaceTablet: tablet bbf3d70de6b143e88c9dfc83f7648e8f deleted and replaced by tablet 23b3e64e87dd4302a68a8cff81a03da1
I20261118 14:58:16.784067 1827 raft_consensus.cc:386] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.784179 1827 raft_consensus.cc:748] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.784473 1827 consensus_queue.cc:261] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
14:58:16.786 [HMS - ERROR - pool-6-thread-5] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_155ca12a699d4393b30979580a04864a already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at jdk.internal.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
I20261118 14:58:16.787851 1827 ts_tablet_manager.cc:1480] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
W20261118 14:58:16.788726 1470 catalog_manager.cc:2211] Already present: failed to create HMS catalog entry for table [id=e883b8c97bfd4068b36badcace826675]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_155ca12a699d4393b30979580a04864a already exists)
I20261118 14:58:16.790133 1470 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]). Expected version 1 got 0
I20261118 14:58:16.790282 1827 tablet_bootstrap.cc:492] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:16.792212 1827 tablet_bootstrap.cc:654] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.795212 1188 tablet_service.cc:1500] Processing CreateTablet for tablet 23b3e64e87dd4302a68a8cff81a03da1 (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.795742 1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b3e64e87dd4302a68a8cff81a03da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.796913 1826 ts_tablet_manager.cc:1975] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20261118 14:58:16.797068 1826 log.cc:1199] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/bbf3d70de6b143e88c9dfc83f7648e8f
I20261118 14:58:16.797474 1826 ts_tablet_manager.cc:1996] T bbf3d70de6b143e88c9dfc83f7648e8f P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.798653 1608 raft_consensus.cc:499] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:16.798811 1608 raft_consensus.cc:521] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.798959 1053 tablet_service.cc:1500] Processing CreateTablet for tablet 23b3e64e87dd4302a68a8cff81a03da1 (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.799379 1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b3e64e87dd4302a68a8cff81a03da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.799456 1608 leader_election.cc:290] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:16.800061 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "522efb108dd140b8a3fd476d202a073e" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:16.800242 1211 raft_consensus.cc:2490] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:16.800734 1012 leader_election.cc:304] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:16.801076 1608 raft_consensus.cc:2826] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:16.801177 1608 raft_consensus.cc:499] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:16.801281 1608 raft_consensus.cc:3096] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:16.804530 1470 master_service.cc:965] ReplaceTablet: received request to replace tablet f66c4e0f06f043108a6451666a80a5f9 from {username='slave'} at 127.0.0.1:50956
I20261118 14:58:16.804812 1608 raft_consensus.cc:521] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.805895 1608 leader_election.cc:290] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:16.806001 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "522efb108dd140b8a3fd476d202a073e" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:16.806191 1211 raft_consensus.cc:3096] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:16.806870 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:16.808835 1211 raft_consensus.cc:2490] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:16.809475 1012 leader_election.cc:304] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:16.809803 1608 raft_consensus.cc:2826] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:16.814670 1477 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]). Expected version 1 got 0
I20261118 14:58:16.815145 1470 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet f66c4e0f06f043108a6451666a80a5f9
I20261118 14:58:16.816439 1477 master_service.cc:965] ReplaceTablet: received request to replace tablet 86c2240a7e3d415e92776000511c7557 from {username='slave'} at 127.0.0.1:50956
I20261118 14:58:16.816577 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet f66c4e0f06f043108a6451666a80a5f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b5222f6dc7f4757adbf34c3f6a2729e) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.817080 1826 tablet_replica.cc:333] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.817306 1826 raft_consensus.cc:2261] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.817615 1826 raft_consensus.cc:2290] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.818228 1826 ts_tablet_manager.cc:1962] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.818598 1835 tablet_bootstrap.cc:492] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:16.820506 1835 tablet_bootstrap.cc:654] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.821815 1831 ts_tablet_manager.cc:1975] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20261118 14:58:16.821990 1831 log.cc:1199] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/bbf3d70de6b143e88c9dfc83f7648e8f
I20261118 14:58:16.822804 1831 ts_tablet_manager.cc:1996] T bbf3d70de6b143e88c9dfc83f7648e8f P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.823812 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:16.827351 1828 ts_tablet_manager.cc:1962] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.827766 1835 tablet_bootstrap.cc:492] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:16.827915 1835 ts_tablet_manager.cc:1449] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.001s
I20261118 14:58:16.828639 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "522efb108dd140b8a3fd476d202a073e" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:16.828651 1835 raft_consensus.cc:360] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.828852 1835 raft_consensus.cc:386] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.828843 938 raft_consensus.cc:2490] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:16.828931 1835 raft_consensus.cc:748] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.829180 1835 consensus_queue.cc:261] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.830125 915 tablet_service.cc:1500] Processing CreateTablet for tablet 23b3e64e87dd4302a68a8cff81a03da1 (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.830857 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23b3e64e87dd4302a68a8cff81a03da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.831388 1835 ts_tablet_manager.cc:1480] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20261118 14:58:16.831682 1835 tablet_bootstrap.cc:492] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:16.833256 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "522efb108dd140b8a3fd476d202a073e" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:16.833420 938 raft_consensus.cc:3096] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:16.834640 1053 tablet_service.cc:1500] Processing CreateTablet for tablet 1b5222f6dc7f4757adbf34c3f6a2729e (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.835147 1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5222f6dc7f4757adbf34c3f6a2729e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.839283 917 tablet_service.cc:1547] Processing DeleteTablet for tablet f66c4e0f06f043108a6451666a80a5f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b5222f6dc7f4757adbf34c3f6a2729e) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.839744 919 tablet_service.cc:1500] Processing CreateTablet for tablet 1b5222f6dc7f4757adbf34c3f6a2729e (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.840240 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5222f6dc7f4757adbf34c3f6a2729e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.840965 1470 catalog_manager.cc:6936] ReplaceTablet: tablet f66c4e0f06f043108a6451666a80a5f9 deleted and replaced by tablet 1b5222f6dc7f4757adbf34c3f6a2729e
I20261118 14:58:16.845940 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet f66c4e0f06f043108a6451666a80a5f9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1b5222f6dc7f4757adbf34c3f6a2729e) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.846637 1831 tablet_replica.cc:333] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.846917 1831 raft_consensus.cc:2261] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.847363 1831 raft_consensus.cc:2290] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.848050 1831 ts_tablet_manager.cc:1962] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.851758 1190 tablet_service.cc:1500] Processing CreateTablet for tablet 1b5222f6dc7f4757adbf34c3f6a2729e (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.852425 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b5222f6dc7f4757adbf34c3f6a2729e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.852815 1477 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet 86c2240a7e3d415e92776000511c7557
I20261118 14:58:16.856164 938 raft_consensus.cc:2490] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:16.859495 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 86c2240a7e3d415e92776000511c7557 with delete_type TABLET_DATA_DELETED (replaced by tablet 03faa53951af43f09a774e20db591133) from {username='slave'} at 127.0.0.1:49492
I20261118 14:58:16.860816 917 tablet_service.cc:1547] Processing DeleteTablet for tablet 86c2240a7e3d415e92776000511c7557 with delete_type TABLET_DATA_DELETED (replaced by tablet 03faa53951af43f09a774e20db591133) from {username='slave'} at 127.0.0.1:54860
I20261118 14:58:16.862421 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet 86c2240a7e3d415e92776000511c7557 with delete_type TABLET_DATA_DELETED (replaced by tablet 03faa53951af43f09a774e20db591133) from {username='slave'} at 127.0.0.1:60878
I20261118 14:58:16.862975 1835 tablet_bootstrap.cc:654] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
14:58:16.862 [HMS - ERROR - pool-6-thread-5] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_9021a49aff8546eca95c57bde20822b3 already exists)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
at jdk.internal.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
W20261118 14:58:16.863868 1472 catalog_manager.cc:2211] Already present: failed to create HMS catalog entry for table [id=fa106faa976b4dabbdcfdc0bc708ee1c]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_9021a49aff8546eca95c57bde20822b3 already exists)
I20261118 14:58:16.864440 1472 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_4dba69207cca45529e828162200c13f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:16.863785 1828 ts_tablet_manager.cc:1975] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20261118 14:58:16.864857 1828 log.cc:1199] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/bbf3d70de6b143e88c9dfc83f7648e8f
I20261118 14:58:16.866190 1829 tablet_bootstrap.cc:654] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.867862 1828 ts_tablet_manager.cc:1996] T bbf3d70de6b143e88c9dfc83f7648e8f P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.870193 1828 tablet_replica.cc:333] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.870404 1828 raft_consensus.cc:2261] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.870615 1828 raft_consensus.cc:2290] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.870702 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet bbf3d70de6b143e88c9dfc83f7648e8f (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:16.871129 1828 ts_tablet_manager.cc:1962] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.871842 1831 ts_tablet_manager.cc:1975] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.871946 1831 log.cc:1199] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/f66c4e0f06f043108a6451666a80a5f9
I20261118 14:58:16.872288 1831 ts_tablet_manager.cc:1996] T f66c4e0f06f043108a6451666a80a5f9 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.873268 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet f66c4e0f06f043108a6451666a80a5f9 (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]) successfully deleted
I20261118 14:58:16.876394 1831 tablet_replica.cc:333] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:16.876618 1831 raft_consensus.cc:2261] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.876801 1831 raft_consensus.cc:2290] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.878192 915 tablet_service.cc:1500] Processing CreateTablet for tablet 03faa53951af43f09a774e20db591133 (DEFAULT_TABLE table=default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.878736 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03faa53951af43f09a774e20db591133. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.878619 1831 ts_tablet_manager.cc:1962] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.879678 1188 tablet_service.cc:1500] Processing CreateTablet for tablet 03faa53951af43f09a774e20db591133 (DEFAULT_TABLE table=default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.880144 1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03faa53951af43f09a774e20db591133. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.884364 1857 raft_consensus.cc:499] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:16.884531 1857 raft_consensus.cc:521] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.884970 1053 tablet_service.cc:1500] Processing CreateTablet for tablet 03faa53951af43f09a774e20db591133 (DEFAULT_TABLE table=default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:16.885501 1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03faa53951af43f09a774e20db591133. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:16.885766 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c8354c983bc5487e9c94202616d222c6" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:16.885999 938 raft_consensus.cc:2490] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:16.886534 1010 leader_election.cc:304] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:16.889118 1827 tablet_bootstrap.cc:492] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:16.889308 1827 ts_tablet_manager.cc:1449] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.099s user 0.010s sys 0.004s
I20261118 14:58:16.890202 1827 raft_consensus.cc:360] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.890496 1827 raft_consensus.cc:386] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.890642 1827 raft_consensus.cc:748] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.891053 1827 consensus_queue.cc:261] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.891595 1831 ts_tablet_manager.cc:1975] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.891753 1831 log.cc:1199] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/86c2240a7e3d415e92776000511c7557
I20261118 14:58:16.892163 1831 ts_tablet_manager.cc:1996] T 86c2240a7e3d415e92776000511c7557 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:16.892771 1477 catalog_manager.cc:6936] ReplaceTablet: tablet 86c2240a7e3d415e92776000511c7557 deleted and replaced by tablet 03faa53951af43f09a774e20db591133
I20261118 14:58:16.893205 1857 leader_election.cc:290] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:16.893342 1465 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:16.893484 1857 raft_consensus.cc:2826] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:16.893265 1477 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_053874e1edfe4b8eac52b4ba5f1b1870"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:16.893741 1857 raft_consensus.cc:499] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:16.893975 1857 raft_consensus.cc:3096] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:16.894431 1470 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet 23b3e64e87dd4302a68a8cff81a03da1 (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]). Expected version 2 got 0
I20261118 14:58:16.895018 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c8354c983bc5487e9c94202616d222c6" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:16.895246 1211 raft_consensus.cc:2490] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:16.895802 1835 tablet_bootstrap.cc:492] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:16.895970 1835 ts_tablet_manager.cc:1449] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.064s user 0.001s sys 0.009s
W20261118 14:58:16.896130 1470 catalog_manager.cc:6040] Failed to send AlterTable RPC for tablet 23b3e64e87dd4302a68a8cff81a03da1 (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 23b3e64e87dd4302a68a8cff81a03da1 consensus state has no leader
I20261118 14:58:16.896677 1835 raft_consensus.cc:360] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.896878 1835 raft_consensus.cc:386] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.896960 1835 raft_consensus.cc:748] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.897181 1835 consensus_queue.cc:261] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.898685 1835 ts_tablet_manager.cc:1480] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:16.898970 1835 tablet_bootstrap.cc:492] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:16.899920 1857 raft_consensus.cc:521] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.897311 1470 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_d24daba84f8c4c94824da319564f0453"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:16.901770 1835 tablet_bootstrap.cc:654] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.903235 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c8354c983bc5487e9c94202616d222c6" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:16.903458 938 raft_consensus.cc:3096] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:16.905184 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c8354c983bc5487e9c94202616d222c6" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:16.905372 1211 raft_consensus.cc:3096] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:16.907794 1826 ts_tablet_manager.cc:1975] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.907912 1826 log.cc:1199] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/f66c4e0f06f043108a6451666a80a5f9
I20261118 14:58:16.908301 1826 ts_tablet_manager.cc:1996] T f66c4e0f06f043108a6451666a80a5f9 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.908375 1211 raft_consensus.cc:2490] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:16.908803 1826 tablet_replica.cc:333] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:16.909013 1826 raft_consensus.cc:2261] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:16.909036 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet f66c4e0f06f043108a6451666a80a5f9 (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]) successfully deleted
I20261118 14:58:16.913280 938 raft_consensus.cc:2490] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:16.913450 1828 ts_tablet_manager.cc:1975] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.913570 1828 log.cc:1199] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/f66c4e0f06f043108a6451666a80a5f9
I20261118 14:58:16.913981 1828 ts_tablet_manager.cc:1996] T f66c4e0f06f043108a6451666a80a5f9 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.914673 1835 tablet_bootstrap.cc:492] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:16.915454 1835 ts_tablet_manager.cc:1449] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20261118 14:58:16.915705 1829 tablet_bootstrap.cc:492] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:16.916501 1835 raft_consensus.cc:360] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.916807 1829 ts_tablet_manager.cc:1449] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.145s user 0.001s sys 0.007s
I20261118 14:58:16.916790 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet f66c4e0f06f043108a6451666a80a5f9 (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]) successfully deleted
I20261118 14:58:16.917834 1835 raft_consensus.cc:386] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.917640 1829 raft_consensus.cc:360] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.919030 1829 raft_consensus.cc:386] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.909199 1012 leader_election.cc:304] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:16.909320 1857 leader_election.cc:290] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:16.919620 1826 raft_consensus.cc:2290] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.914835 1827 ts_tablet_manager.cc:1480] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.025s user 0.000s sys 0.002s
I20261118 14:58:16.920116 1827 tablet_bootstrap.cc:492] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:16.919411 1855 raft_consensus.cc:2826] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:16.920622 1855 raft_consensus.cc:705] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:16.920928 1855 consensus_queue.cc:238] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.919085 1608 raft_consensus.cc:705] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:16.922139 1608 consensus_queue.cc:238] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:16.925091 1827 tablet_bootstrap.cc:654] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.919167 1829 raft_consensus.cc:748] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.920444 1826 ts_tablet_manager.cc:1962] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.925503 1861 consensus_queue.cc:1049] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:16.925892 1829 consensus_queue.cc:261] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.926667 1835 raft_consensus.cc:748] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.927771 1861 consensus_queue.cc:1049] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:16.928309 1829 ts_tablet_manager.cc:1480] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20261118 14:58:16.927033 1835 consensus_queue.cc:261] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.929731 1835 ts_tablet_manager.cc:1480] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20261118 14:58:16.930181 1835 tablet_bootstrap.cc:492] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:16.920424 1828 tablet_replica.cc:333] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:16.931102 1828 raft_consensus.cc:2261] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:16.931304 1828 raft_consensus.cc:2290] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:16.931874 1828 ts_tablet_manager.cc:1962] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:16.934051 1835 tablet_bootstrap.cc:654] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.935478 1829 tablet_bootstrap.cc:492] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:16.937577 1829 tablet_bootstrap.cc:654] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:16.942267 1835 tablet_bootstrap.cc:492] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:16.943380 1835 ts_tablet_manager.cc:1449] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20261118 14:58:16.943930 1828 ts_tablet_manager.cc:1975] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.956931 1861 consensus_queue.cc:1049] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:16.984083 1826 ts_tablet_manager.cc:1975] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:16.984352 1471 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:16.991110 1479 catalog_manager.cc:5840] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c4fe5e44448d98db1eed6329d150d (127.0.72.130). New cstate: current_term: 1 leader_uuid: "245c4fe5e44448d98db1eed6329d150d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: UNKNOWN } } }
I20261118 14:58:16.991955 1828 log.cc:1199] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/86c2240a7e3d415e92776000511c7557
I20261118 14:58:16.944913 1835 raft_consensus.cc:360] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.992106 1835 raft_consensus.cc:386] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.992194 1835 raft_consensus.cc:748] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.992427 1828 ts_tablet_manager.cc:1996] T 86c2240a7e3d415e92776000511c7557 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:16.992455 1835 consensus_queue.cc:261] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.993315 1835 ts_tablet_manager.cc:1480] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.049s user 0.001s sys 0.000s
I20261118 14:58:16.992733 1826 log.cc:1199] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/86c2240a7e3d415e92776000511c7557
I20261118 14:58:16.993889 1826 ts_tablet_manager.cc:1996] T 86c2240a7e3d415e92776000511c7557 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:16.994155 1479 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:16.994688 1463 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:16.994813 1861 consensus_queue.cc:1049] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:16.995054 1466 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 86c2240a7e3d415e92776000511c7557 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:16.996389 1829 tablet_bootstrap.cc:492] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:16.996547 1829 ts_tablet_manager.cc:1449] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.061s user 0.000s sys 0.004s
I20261118 14:58:16.997323 1829 raft_consensus.cc:360] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:16.998549 1829 raft_consensus.cc:386] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:16.998627 1829 raft_consensus.cc:748] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:16.998852 1829 consensus_queue.cc:261] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.000082 1829 ts_tablet_manager.cc:1480] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:17.000397 1829 tablet_bootstrap.cc:492] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:17.002216 1829 tablet_bootstrap.cc:654] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
F20261118 14:58:17.002419 1531 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 36, previous 36; dumping first 4 out of 5 received events: NotificationEvent(eventId=34, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_4789eba8c66842f6ae326364062589fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_4789eba8c66842f6ae326364062589fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4789eba8c66842f6ae326364062589fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4789eba8c66842f6ae326364062589fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4789eba8c66842f6ae326364062589fb\",\"kudu.table_id\":\"0ce3ba58cd7440a9b27427e7713af331\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=35, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_7171b2f0b6ea470393bf9aa28246cf65, message={"server":"","servicePrincipal":"","db":"default","table":"table_7171b2f0b6ea470393bf9aa28246cf65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_7171b2f0b6ea470393bf9aa28246cf65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_7171b2f0b6ea470393bf9aa28246cf65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_7171b2f0b6ea470393bf9aa28246cf65\",\"kudu.table_id\":\"cdc2cda186804cd39b0d320f08e91df8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_4d2c8b2bcb994782a8bbf4b38ca2f7af, message={"server":"","servicePrincipal":"","db":"default","table":"table_4d2c8b2bcb994782a8bbf4b38ca2f7af","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af\",\"kudu.table_id\":\"d65d9ba97cda44e3bdff3add1fec1ec8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_36b9017c9ae24ecdb8ae81bf85341052, message={"server":"","servicePrincipal":"","db":"default","table":"table_36b9017c9ae24ecdb8ae81bf85341052","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_36b9017c9ae24ecdb8ae81bf85341052\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_36b9017c9ae24ecdb8ae81bf85341052\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_36b9017c9ae24ecdb8ae81bf85341052\",\"kudu.table_id\":\"378bd168f216477992c1b7bb33791778\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fcdee7e4d5d google::LogMessage::Fail() at ??:0
I20261118 14:58:17.009021 1827 tablet_bootstrap.cc:492] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:17.009172 1827 ts_tablet_manager.cc:1449] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.089s user 0.012s sys 0.006s
@ 0x7fcdee7e8b23 google::LogMessage::SendToLog() at ??:0
I20261118 14:58:17.009905 1827 raft_consensus.cc:360] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.010108 1827 raft_consensus.cc:386] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:17.010171 1827 raft_consensus.cc:748] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:17.010393 1827 consensus_queue.cc:261] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.012791 1479 catalog_manager.cc:5840] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c4fe5e44448d98db1eed6329d150d (127.0.72.130). New cstate: current_term: 1 leader_uuid: "245c4fe5e44448d98db1eed6329d150d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } }
I20261118 14:58:17.013155 1479 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]). Expected version 1 got 0
I20261118 14:58:17.013338 1479 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:17.013549 1479 catalog_manager.cc:5940] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343) does not have the latest schema for tablet 1b5222f6dc7f4757adbf34c3f6a2729e (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]). Expected version 1 got 0
@ 0x7fcdee7e475c google::LogMessage::Flush() at ??:0
I20261118 14:58:17.015483 1827 ts_tablet_manager.cc:1480] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20261118 14:58:17.015818 1827 tablet_bootstrap.cc:492] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
@ 0x7fcdee7e49a1 google::LogMessage::~LogMessage() at ??:0
I20261118 14:58:17.017537 1827 tablet_bootstrap.cc:654] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
W20261118 14:58:17.025597 1479 catalog_manager.cc:6040] Failed to send AlterTable RPC for tablet 1b5222f6dc7f4757adbf34c3f6a2729e (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 1b5222f6dc7f4757adbf34c3f6a2729e consensus state has no leader
@ 0x7fce03c649a4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 14:58:17.026980 1479 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]). Expected version 1 got 0
I20261118 14:58:17.027220 1479 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]). Expected version 1 got 0
I20261118 14:58:17.028235 1479 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50956:
name: "default.Table_4789eba8c66842f6ae326364062589fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:17.032399 1827 tablet_bootstrap.cc:492] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:17.032547 1827 ts_tablet_manager.cc:1449] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
@ 0x7fce03c63251 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 14:58:17.033352 1827 raft_consensus.cc:360] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.033540 1827 raft_consensus.cc:386] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:17.033629 1827 raft_consensus.cc:748] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:17.033854 1827 consensus_queue.cc:261] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
@ 0x7fce03c6aecc kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 14:58:17.037937 1827 ts_tablet_manager.cc:1480] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20261118 14:58:17.038223 1827 tablet_bootstrap.cc:492] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:17.040386 1827 tablet_bootstrap.cc:654] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:17.040802 1689 raft_consensus.cc:499] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:17.040946 1689 raft_consensus.cc:521] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.041678 1689 leader_election.cc:290] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
@ 0x7fce03c6ad49 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x372c7d std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 14:58:17.042819 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1b5222f6dc7f4757adbf34c3f6a2729e" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:17.042939 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1b5222f6dc7f4757adbf34c3f6a2729e" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:17.043040 1075 raft_consensus.cc:2490] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:17.043126 938 raft_consensus.cc:2490] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:17.043515 1148 leader_election.cc:304] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:17.043948 1689 raft_consensus.cc:2826] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:17.044060 1689 raft_consensus.cc:499] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:17.044131 1689 raft_consensus.cc:3096] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.046890 1829 tablet_bootstrap.cc:492] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:17.047029 1829 ts_tablet_manager.cc:1449] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20261118 14:58:17.046939 1689 raft_consensus.cc:521] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
@ 0x7fcdefc2ddc9 kudu::Thread::SuperviseThread() at ??:0
I20261118 14:58:17.047768 1829 raft_consensus.cc:360] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.047926 1829 raft_consensus.cc:386] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:17.047997 1829 raft_consensus.cc:748] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:17.048566 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1b5222f6dc7f4757adbf34c3f6a2729e" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:17.048722 938 raft_consensus.cc:3096] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
@ 0x7fcdf33086db start_thread at ??:0
@ 0x7fcdebd7861f clone at ??:0
I20261118 14:58:17.048187 1829 consensus_queue.cc:261] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.051769 1829 ts_tablet_manager.cc:1480] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20261118 14:58:17.051792 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1b5222f6dc7f4757adbf34c3f6a2729e" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:17.051962 1075 raft_consensus.cc:3096] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.052587 1829 tablet_bootstrap.cc:492] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:17.054206 1829 tablet_bootstrap.cc:654] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:17.055335 1075 raft_consensus.cc:2490] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:17.056011 1148 leader_election.cc:304] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:17.059190 1689 leader_election.cc:290] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:17.059370 1689 raft_consensus.cc:2826] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:17.059492 1689 raft_consensus.cc:705] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:17.059741 1689 consensus_queue.cc:238] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.061041 938 raft_consensus.cc:2490] T 1b5222f6dc7f4757adbf34c3f6a2729e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:17.070328 1829 tablet_bootstrap.cc:492] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:17.070497 1829 ts_tablet_manager.cc:1449] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.006s
I20261118 14:58:17.071216 1829 raft_consensus.cc:360] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.071591 1829 raft_consensus.cc:386] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:17.071662 1829 raft_consensus.cc:748] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:17.071872 1829 consensus_queue.cc:261] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.073868 1829 ts_tablet_manager.cc:1480] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:17.075834 1608 raft_consensus.cc:499] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:17.075994 1608 raft_consensus.cc:521] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.076943 1608 leader_election.cc:290] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:17.077539 1827 tablet_bootstrap.cc:492] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:17.077710 1827 ts_tablet_manager.cc:1449] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.040s user 0.005s sys 0.005s
I20261118 14:58:17.078001 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "96011b926cca4e4bb2f66fc7944201f7" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:17.078189 938 raft_consensus.cc:2490] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:17.078401 1827 raft_consensus.cc:360] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.078608 1827 raft_consensus.cc:386] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:17.078644 1010 leader_election.cc:304] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:17.078684 1827 raft_consensus.cc:748] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:17.078895 1608 raft_consensus.cc:2826] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:17.079003 1608 raft_consensus.cc:499] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:17.078935 1827 consensus_queue.cc:261] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.079083 1608 raft_consensus.cc:3096] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.079562 1827 ts_tablet_manager.cc:1480] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:17.083734 1608 raft_consensus.cc:521] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.084666 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "96011b926cca4e4bb2f66fc7944201f7" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:17.084826 938 raft_consensus.cc:3096] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.086817 1883 consensus_queue.cc:1049] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20261118 14:58:17.087466 938 raft_consensus.cc:2490] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:17.087863 1010 leader_election.cc:304] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:17.084472 1608 leader_election.cc:290] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:17.088173 1857 raft_consensus.cc:2826] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:17.088290 1857 raft_consensus.cc:705] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:17.088557 1857 consensus_queue.cc:238] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.089421 1886 consensus_queue.cc:1049] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:17.094595 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "96011b926cca4e4bb2f66fc7944201f7" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:17.094784 1211 raft_consensus.cc:2490] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:17.095407 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "96011b926cca4e4bb2f66fc7944201f7" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:17.095594 1211 raft_consensus.cc:3096] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.100237 1211 raft_consensus.cc:2490] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:17.112815 1861 consensus_queue.cc:1049] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:17.115593 1861 consensus_queue.cc:1049] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:17.139065 1857 raft_consensus.cc:499] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:17.139262 1857 raft_consensus.cc:521] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.139906 1857 leader_election.cc:290] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:17.140390 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cafee7e386434b3c85542d7b6c5024ed" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:17.140424 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cafee7e386434b3c85542d7b6c5024ed" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:17.140589 938 raft_consensus.cc:2490] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:17.140616 1211 raft_consensus.cc:2490] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:17.141031 1010 leader_election.cc:304] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:17.141376 1857 raft_consensus.cc:2826] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:17.141464 1857 raft_consensus.cc:499] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:17.141554 1857 raft_consensus.cc:3096] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.144500 1857 raft_consensus.cc:521] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.145402 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cafee7e386434b3c85542d7b6c5024ed" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:17.145587 938 raft_consensus.cc:3096] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.145675 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "cafee7e386434b3c85542d7b6c5024ed" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:17.145828 1211 raft_consensus.cc:3096] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.145466 1857 leader_election.cc:290] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:17.147645 938 raft_consensus.cc:2490] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:17.148056 1010 leader_election.cc:304] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:17.148301 1857 raft_consensus.cc:2826] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:17.148425 1857 raft_consensus.cc:705] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:17.148653 1857 consensus_queue.cc:238] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.149247 1211 raft_consensus.cc:2490] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:17.152189 1855 consensus_queue.cc:1049] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:17.154284 1855 consensus_queue.cc:1049] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:17.240515 1888 raft_consensus.cc:499] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:17.240715 1888 raft_consensus.cc:521] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.241664 1888 leader_election.cc:290] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:17.242235 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "03faa53951af43f09a774e20db591133" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:17.242298 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "03faa53951af43f09a774e20db591133" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:17.242448 1075 raft_consensus.cc:2490] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:17.242584 938 raft_consensus.cc:2490] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:17.242892 1148 leader_election.cc:304] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:17.243217 1888 raft_consensus.cc:2826] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:17.243340 1888 raft_consensus.cc:499] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:17.243450 1888 raft_consensus.cc:3096] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.246093 1888 raft_consensus.cc:521] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.247239 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "03faa53951af43f09a774e20db591133" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:17.247356 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "03faa53951af43f09a774e20db591133" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:17.247445 938 raft_consensus.cc:3096] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.247506 1075 raft_consensus.cc:3096] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.250154 938 raft_consensus.cc:2490] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:17.250404 1888 leader_election.cc:290] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:17.250445 1075 raft_consensus.cc:2490] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:17.250800 1148 leader_election.cc:304] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:17.251098 1888 raft_consensus.cc:2826] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:17.251226 1888 raft_consensus.cc:705] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:17.251533 1888 consensus_queue.cc:238] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.254114 1883 consensus_queue.cc:1049] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:17.262821 1883 consensus_queue.cc:1049] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 14:58:17.290122 1265 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:17.290623 1265 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.189:46707)
W20261118 14:58:17.291049 1265 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.189:46707)
W20261118 14:58:17.291682 1265 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.189:46707)
W20261118 14:58:17.292565 1265 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.189:46707)
I20261118 14:58:17.290786 1686 master_runner.cc:187] Error getting master registration for 127.0.72.189:46707: Network error: recv got EOF from 127.0.72.189:46707 (error 108),
I20261118 14:58:17.297091 1686 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:17.357318 1883 raft_consensus.cc:499] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:17.357499 1883 raft_consensus.cc:521] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.358206 1883 leader_election.cc:290] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:17.358762 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "23b3e64e87dd4302a68a8cff81a03da1" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:17.358775 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "23b3e64e87dd4302a68a8cff81a03da1" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:17.358996 938 raft_consensus.cc:2490] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:17.359007 1075 raft_consensus.cc:2490] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:17.359443 1145 leader_election.cc:304] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:17.359745 1883 raft_consensus.cc:2826] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:17.359825 1883 raft_consensus.cc:499] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:17.359916 1883 raft_consensus.cc:3096] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.361791 1883 raft_consensus.cc:521] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.362290 1883 leader_election.cc:290] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:17.362764 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "23b3e64e87dd4302a68a8cff81a03da1" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:17.362777 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "23b3e64e87dd4302a68a8cff81a03da1" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:17.362944 938 raft_consensus.cc:3096] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.362946 1075 raft_consensus.cc:3096] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:17.365237 1075 raft_consensus.cc:2490] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:17.365547 938 raft_consensus.cc:2490] T 23b3e64e87dd4302a68a8cff81a03da1 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:17.365628 1148 leader_election.cc:304] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:17.365898 1883 raft_consensus.cc:2826] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:17.366134 1883 raft_consensus.cc:705] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:17.366518 1883 consensus_queue.cc:238] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.369392 1892 consensus_queue.cc:1049] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:17.369786 1892 consensus_queue.cc:1049] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:17.528879 1917 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:17.529142 1917 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:17.530055 1917 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:17.531901 1921 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:17.532125 1921 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
W20261118 14:58:17.533118 588 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:17.534101 1921 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
I20261118 14:58:17.534523 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" candidate_term: 4 candidate_status { last_received { term: 3 index: 97 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e" is_pre_election: true
I20261118 14:58:17.534721 613 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b90720bc0b34e4994abf4fc6f9e59e1 in term 3.
I20261118 14:58:17.535221 1713 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters:
W20261118 14:58:17.535195 1714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:17.535539 1921 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20261118 14:58:17.535645 1921 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:17.535740 1921 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 3 FOLLOWER]: Advancing to term 4
W20261118 14:58:17.536940 1714 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:17.538107 1739 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 4 candidate_status { last_received { term: 3 index: 97 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:17.538530 1921 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:17.538859 1739 raft_consensus.cc:2415] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ed65a3719634e519d783540afd6189e in current term 4: Already voted for candidate 9b90720bc0b34e4994abf4fc6f9e59e1 in this term.
I20261118 14:58:17.539147 1921 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 4 election: Requested vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
I20261118 14:58:17.539454 613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" candidate_term: 4 candidate_status { last_received { term: 3 index: 97 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e"
I20261118 14:58:17.539566 589 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:17.539624 613 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 3 FOLLOWER]: Advancing to term 4
W20261118 14:58:17.540573 1714 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:17.541666 613 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b90720bc0b34e4994abf4fc6f9e59e1 in term 4.
I20261118 14:58:17.541811 1917 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 14:58:17.542037 1713 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 7c9ef96ff074450a879e66787486386d
I20261118 14:58:17.542300 1921 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 4 FOLLOWER]: Leader election won for term 4
I20261118 14:58:17.542625 1921 raft_consensus.cc:705] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 4 LEADER]: Becoming Leader. State: Replica: 9b90720bc0b34e4994abf4fc6f9e59e1, State: Running, Role: LEADER
I20261118 14:58:17.543005 1921 consensus_queue.cc:238] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 3.97, Last appended by leader: 97, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:17.544266 1925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b90720bc0b34e4994abf4fc6f9e59e1. Latest consensus state: current_term: 4 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:17.544701 1925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:17.545531 613 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Refusing update from remote peer 9b90720bc0b34e4994abf4fc6f9e59e1: Log matching property violated. Preceding OpId in replica: term: 3 index: 97. Preceding OpId from leader: term: 4 index: 98. (index mismatch)
I20261118 14:58:17.545614 1928 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 14:58:17.546110 1925 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 97, Time since last communication: 0.000s
W20261118 14:58:17.546695 1714 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 -> Peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Couldn't send request to peer 7c9ef96ff074450a879e66787486386d. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:17.547616 1917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b90720bc0b34e4994abf4fc6f9e59e1. Latest consensus state: current_term: 4 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:17.547784 1917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:17.548933 1928 catalog_manager.cc:694] Loaded metadata for table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:17.549639 1928 catalog_manager.cc:694] Loaded metadata for table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:17.549677 1917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:17.549850 1917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:17.550091 1928 catalog_manager.cc:694] Loaded metadata for table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:17.550547 1928 catalog_manager.cc:694] Loaded metadata for table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]
I20261118 14:58:17.551005 1921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:17.551203 1921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:17.551230 1928 catalog_manager.cc:694] Loaded metadata for table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:17.551664 1928 catalog_manager.cc:694] Loaded metadata for table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:17.552320 1928 catalog_manager.cc:694] Loaded metadata for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:17.552750 1928 catalog_manager.cc:694] Loaded metadata for table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]
I20261118 14:58:17.553102 1928 catalog_manager.cc:694] Loaded metadata for table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]
I20261118 14:58:17.553382 1928 catalog_manager.cc:694] Loaded metadata for table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]
I20261118 14:58:17.553783 1928 catalog_manager.cc:694] Loaded metadata for table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]
I20261118 14:58:17.555612 1928 tablet_loader.cc:96] loaded metadata for tablet 03faa53951af43f09a774e20db591133 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b])
I20261118 14:58:17.556245 1928 tablet_loader.cc:96] loaded metadata for tablet 1b5222f6dc7f4757adbf34c3f6a2729e (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28])
I20261118 14:58:17.556772 1928 tablet_loader.cc:96] loaded metadata for tablet 23b3e64e87dd4302a68a8cff81a03da1 (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d])
I20261118 14:58:17.557021 1928 tablet_loader.cc:96] loaded metadata for tablet 309f067161fe469dbb29d14c16139058 (table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15])
I20261118 14:58:17.557253 1928 tablet_loader.cc:96] loaded metadata for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0])
I20261118 14:58:17.558740 1928 tablet_loader.cc:96] loaded metadata for tablet 96011b926cca4e4bb2f66fc7944201f7 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8])
I20261118 14:58:17.559440 1928 tablet_loader.cc:96] loaded metadata for tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7])
I20261118 14:58:17.559719 1928 tablet_loader.cc:96] loaded metadata for tablet cafee7e386434b3c85542d7b6c5024ed (table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8])
I20261118 14:58:17.560492 1928 tablet_loader.cc:96] loaded metadata for tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6])
I20261118 14:58:17.560781 1928 tablet_loader.cc:96] loaded metadata for tablet e47d7d10a5c741ecb30ec701c76b27a3 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f])
I20261118 14:58:17.561347 1928 tablet_loader.cc:96] loaded metadata for tablet f87ee223b5a246c58e9b2eca5740d08c (table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627])
I20261118 14:58:17.561591 1928 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:17.562742 1928 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:17.562819 1928 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:17.564101 1928 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:17.565052 1928 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Loaded TSK: 0
I20261118 14:58:17.565825 1928 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:17.566833 1928 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:17.567888 1928 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 33
I20261118 14:58:17.589615 1729 master_service.cc:965] ReplaceTablet: received request to replace tablet 96011b926cca4e4bb2f66fc7944201f7 from {username='slave'} at 127.0.0.1:39150
I20261118 14:58:17.589699 1728 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39150:
name: "default.Table_36b9017c9ae24ecdb8ae81bf85341052"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:17.590572 1727 master_service.cc:965] ReplaceTablet: received request to replace tablet cafee7e386434b3c85542d7b6c5024ed from {username='slave'} at 127.0.0.1:39150
I20261118 14:58:17.590636 1726 master_service.cc:965] ReplaceTablet: received request to replace tablet 309f067161fe469dbb29d14c16139058 from {username='slave'} at 127.0.0.1:39150
I20261118 14:58:17.590688 1723 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39150:
name: "default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:17.590760 1724 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39150:
name: "default.Table_5c2ab051f54546cab77c75d60b2a8bd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:17.591809 1722 master_service.cc:965] ReplaceTablet: received request to replace tablet c8354c983bc5487e9c94202616d222c6 from {username='slave'} at 127.0.0.1:39150
I20261118 14:58:17.591499 1725 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39150:
name: "default.Table_6c65bee0620c494591ca0d8d3226c6a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:17.595388 1720 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39150:
table { table_name: "default.Table_4d2c8b2bcb994782a8bbf4b38ca2f7af" } new_table_name: "default.Table_b56407e6850a466082c3b8d7c0dae8d0" modify_external_catalogs: true
I20261118 14:58:17.593070 1721 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39150:
name: "default.Table_d0b14f4adfe3441c88447b2fc1e2d506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20261118 14:58:17.599713 1783 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 36, previous 36; dumping first 4 out of 5 received events: NotificationEvent(eventId=34, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_4789eba8c66842f6ae326364062589fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_4789eba8c66842f6ae326364062589fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4789eba8c66842f6ae326364062589fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4789eba8c66842f6ae326364062589fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4789eba8c66842f6ae326364062589fb\",\"kudu.table_id\":\"0ce3ba58cd7440a9b27427e7713af331\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=35, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_7171b2f0b6ea470393bf9aa28246cf65, message={"server":"","servicePrincipal":"","db":"default","table":"table_7171b2f0b6ea470393bf9aa28246cf65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_7171b2f0b6ea470393bf9aa28246cf65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_7171b2f0b6ea470393bf9aa28246cf65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_7171b2f0b6ea470393bf9aa28246cf65\",\"kudu.table_id\":\"cdc2cda186804cd39b0d320f08e91df8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_4d2c8b2bcb994782a8bbf4b38ca2f7af, message={"server":"","servicePrincipal":"","db":"default","table":"table_4d2c8b2bcb994782a8bbf4b38ca2f7af","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af\",\"kudu.table_id\":\"d65d9ba97cda44e3bdff3add1fec1ec8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_36b9017c9ae24ecdb8ae81bf85341052, message={"server":"","servicePrincipal":"","db":"default","table":"table_36b9017c9ae24ecdb8ae81bf85341052","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_36b9017c9ae24ecdb8ae81bf85341052\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_36b9017c9ae24ecdb8ae81bf85341052\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_36b9017c9ae24ecdb8ae81bf85341052\",\"kudu.table_id\":\"378bd168f216477992c1b7bb33791778\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7fda81951d5d google::LogMessage::Fail() at ??:0
@ 0x7fda81955b23 google::LogMessage::SendToLog() at ??:0
I20261118 14:58:17.605573 1729 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet 96011b926cca4e4bb2f66fc7944201f7
@ 0x7fda8195175c google::LogMessage::Flush() at ??:0
@ 0x7fda819519a1 google::LogMessage::~LogMessage() at ??:0
I20261118 14:58:17.613793 1729 catalog_manager.cc:6936] ReplaceTablet: tablet 96011b926cca4e4bb2f66fc7944201f7 deleted and replaced by tablet a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:17.614876 1729 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39150:
table { table_name: "default.Table_7171b2f0b6ea470393bf9aa28246cf65" } new_table_name: "default.Table_de4cb9add02543cb98fda40cb17f0587" modify_external_catalogs: true
@ 0x7fda96dd19a4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 14:58:17.626817 1727 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet cafee7e386434b3c85542d7b6c5024ed
I20261118 14:58:17.627075 1726 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet 309f067161fe469dbb29d14c16139058
I20261118 14:58:17.627405 1727 catalog_manager.cc:6936] ReplaceTablet: tablet cafee7e386434b3c85542d7b6c5024ed deleted and replaced by tablet 77e643a760294dacbfca6305f1811bbd
I20261118 14:58:17.627538 1726 catalog_manager.cc:6936] ReplaceTablet: tablet 309f067161fe469dbb29d14c16139058 deleted and replaced by tablet 42d8a3bab2694c3e83da6a8642d6d911
I20261118 14:58:17.627696 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet 96011b926cca4e4bb2f66fc7944201f7 with delete_type TABLET_DATA_DELETED (replaced by tablet a78ecb3a620242f4a16128b0d456c352) from {username='slave'} at 127.0.0.1:60884
I20261118 14:58:17.631552 1189 tablet_service.cc:1547] Processing DeleteTablet for tablet 309f067161fe469dbb29d14c16139058 with delete_type TABLET_DATA_DELETED (replaced by tablet 42d8a3bab2694c3e83da6a8642d6d911) from {username='slave'} at 127.0.0.1:60884
I20261118 14:58:17.633778 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet cafee7e386434b3c85542d7b6c5024ed with delete_type TABLET_DATA_DELETED (replaced by tablet 77e643a760294dacbfca6305f1811bbd) from {username='slave'} at 127.0.0.1:60884
@ 0x7fda96dd0251 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 14:58:17.639025 1942 tablet_replica.cc:333] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:17.639650 1942 raft_consensus.cc:2261] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:17.639863 1942 raft_consensus.cc:2290] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:17.640446 915 tablet_service.cc:1547] Processing DeleteTablet for tablet 96011b926cca4e4bb2f66fc7944201f7 with delete_type TABLET_DATA_DELETED (replaced by tablet a78ecb3a620242f4a16128b0d456c352) from {username='slave'} at 127.0.0.1:54862
I20261118 14:58:17.640597 1942 ts_tablet_manager.cc:1962] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:17.641093 1943 tablet_replica.cc:333] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:17.641319 915 tablet_service.cc:1547] Processing DeleteTablet for tablet cafee7e386434b3c85542d7b6c5024ed with delete_type TABLET_DATA_DELETED (replaced by tablet 77e643a760294dacbfca6305f1811bbd) from {username='slave'} at 127.0.0.1:54862
I20261118 14:58:17.641724 915 tablet_service.cc:1547] Processing DeleteTablet for tablet 309f067161fe469dbb29d14c16139058 with delete_type TABLET_DATA_DELETED (replaced by tablet 42d8a3bab2694c3e83da6a8642d6d911) from {username='slave'} at 127.0.0.1:54862
I20261118 14:58:17.642771 1726 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39150:
name: "default.Table_d24daba84f8c4c94824da319564f0453"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:17.643312 1727 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39150:
name: "default.Table_4789eba8c66842f6ae326364062589fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7fda96dd7ecc kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 14:58:17.645777 1943 raft_consensus.cc:2261] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:17.646004 1943 raft_consensus.cc:2290] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:17.646569 1722 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Sending DeleteTablet for 3 replicas of tablet c8354c983bc5487e9c94202616d222c6
I20261118 14:58:17.647130 1722 catalog_manager.cc:6936] ReplaceTablet: tablet c8354c983bc5487e9c94202616d222c6 deleted and replaced by tablet da163bd4e25b4254ab3923af361202ce
@ 0x7fda96dd7d49 std::_Function_handler<>::_M_invoke() at ??:0
I20261118 14:58:17.647513 1722 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39150:
name: "default.Table_053874e1edfe4b8eac52b4ba5f1b1870"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:17.647753 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet c8354c983bc5487e9c94202616d222c6 with delete_type TABLET_DATA_DELETED (replaced by tablet da163bd4e25b4254ab3923af361202ce) from {username='slave'} at 127.0.0.1:60884
I20261118 14:58:17.648041 1942 ts_tablet_manager.cc:1975] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:17.648226 1942 log.cc:1199] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/96011b926cca4e4bb2f66fc7944201f7
I20261118 14:58:17.648660 1942 ts_tablet_manager.cc:1996] T 96011b926cca4e4bb2f66fc7944201f7 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:17.649742 1715 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 96011b926cca4e4bb2f66fc7944201f7 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]) successfully deleted
I20261118 14:58:17.650153 1942 tablet_replica.cc:333] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:17.650398 915 tablet_service.cc:1547] Processing DeleteTablet for tablet c8354c983bc5487e9c94202616d222c6 with delete_type TABLET_DATA_DELETED (replaced by tablet da163bd4e25b4254ab3923af361202ce) from {username='slave'} at 127.0.0.1:54862
I20261118 14:58:17.650914 1188 tablet_service.cc:1500] Processing CreateTablet for tablet a78ecb3a620242f4a16128b0d456c352 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:17.651006 915 tablet_service.cc:1500] Processing CreateTablet for tablet a78ecb3a620242f4a16128b0d456c352 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:17.651288 1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a78ecb3a620242f4a16128b0d456c352. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:17.651468 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a78ecb3a620242f4a16128b0d456c352. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x372c7d std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 14:58:17.655313 1943 ts_tablet_manager.cc:1962] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fda82d9adc9 kudu::Thread::SuperviseThread() at ??:0
I20261118 14:58:17.657586 1942 raft_consensus.cc:2261] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:17.657773 1942 raft_consensus.cc:2290] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:17.658280 1942 ts_tablet_manager.cc:1962] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7fda864756db start_thread at ??:0
@ 0x7fda7eee561f clone at ??:0
I20261118 14:58:17.663560 1943 ts_tablet_manager.cc:1975] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:17.663676 1943 log.cc:1199] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/96011b926cca4e4bb2f66fc7944201f7
I20261118 14:58:17.664104 1943 ts_tablet_manager.cc:1996] T 96011b926cca4e4bb2f66fc7944201f7 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:17.665518 1942 ts_tablet_manager.cc:1975] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:17.665642 1942 log.cc:1199] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/309f067161fe469dbb29d14c16139058
I20261118 14:58:17.666002 1942 ts_tablet_manager.cc:1996] T 309f067161fe469dbb29d14c16139058 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:17.666637 1945 tablet_bootstrap.cc:492] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:17.667165 1942 tablet_replica.cc:333] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:17.667351 1942 raft_consensus.cc:2261] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:17.667538 1942 raft_consensus.cc:2290] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:17.668382 1944 tablet_bootstrap.cc:492] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:17.668462 1942 ts_tablet_manager.cc:1962] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:17.669945 1945 tablet_bootstrap.cc:654] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:17.670595 1944 tablet_bootstrap.cc:654] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:17.671380 1943 tablet_replica.cc:333] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:17.671639 1943 raft_consensus.cc:2261] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:17.671837 1943 raft_consensus.cc:2290] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:17.672705 1943 ts_tablet_manager.cc:1962] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:17.681493 1942 ts_tablet_manager.cc:1975] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:17.681605 1942 log.cc:1199] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/cafee7e386434b3c85542d7b6c5024ed
I20261118 14:58:17.682001 1942 ts_tablet_manager.cc:1996] T cafee7e386434b3c85542d7b6c5024ed P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:17.683063 1942 tablet_replica.cc:333] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:17.683262 1942 raft_consensus.cc:2261] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:17.683414 1942 raft_consensus.cc:2290] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:17.684587 1945 tablet_bootstrap.cc:492] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:17.684700 1942 ts_tablet_manager.cc:1962] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:17.684721 1945 ts_tablet_manager.cc:1449] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.006s
I20261118 14:58:17.685487 1945 raft_consensus.cc:360] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.685670 1945 raft_consensus.cc:386] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:17.685753 1945 raft_consensus.cc:748] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:17.686041 1945 consensus_queue.cc:261] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.690688 1945 ts_tablet_manager.cc:1480] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20261118 14:58:17.691383 1943 ts_tablet_manager.cc:1975] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:17.691525 1943 log.cc:1199] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/cafee7e386434b3c85542d7b6c5024ed
I20261118 14:58:17.691901 1943 ts_tablet_manager.cc:1996] T cafee7e386434b3c85542d7b6c5024ed P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:17.692931 1943 tablet_replica.cc:333] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:17.693121 1943 raft_consensus.cc:2261] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:17.693459 1943 raft_consensus.cc:2290] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:17.693988 1943 ts_tablet_manager.cc:1962] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:17.695420 1944 tablet_bootstrap.cc:492] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:17.695550 1944 ts_tablet_manager.cc:1449] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.027s user 0.014s sys 0.011s
I20261118 14:58:17.695983 1942 ts_tablet_manager.cc:1975] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:17.696113 1942 log.cc:1199] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/c8354c983bc5487e9c94202616d222c6
I20261118 14:58:17.696332 1944 raft_consensus.cc:360] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.696496 1944 raft_consensus.cc:386] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:17.696503 1942 ts_tablet_manager.cc:1996] T c8354c983bc5487e9c94202616d222c6 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:17.696590 1944 raft_consensus.cc:748] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:17.696835 1944 consensus_queue.cc:261] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:17.698273 1944 ts_tablet_manager.cc:1480] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20261118 14:58:17.701996 1943 ts_tablet_manager.cc:1975] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:17.702080 1943 log.cc:1199] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/309f067161fe469dbb29d14c16139058
I20261118 14:58:17.702387 1943 ts_tablet_manager.cc:1996] T 309f067161fe469dbb29d14c16139058 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:17.703395 1943 tablet_replica.cc:333] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:17.703610 1943 raft_consensus.cc:2261] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:17.703769 1943 raft_consensus.cc:2290] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:17.704632 1943 ts_tablet_manager.cc:1962] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:17.711402 1943 ts_tablet_manager.cc:1975] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:17.711565 1943 log.cc:1199] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/c8354c983bc5487e9c94202616d222c6
I20261118 14:58:17.711957 1943 ts_tablet_manager.cc:1996] T c8354c983bc5487e9c94202616d222c6 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:17.747562 290 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu pid 572
I20261118 14:58:17.748665 633 generic_service.cc:196] Checking for leaks (request via RPC)
W20261118 14:58:17.910516 1145 connection.cc:573] server connection from 127.0.0.1:60884 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:17.910584 1266 connection.cc:573] client connection to 127.0.72.188:33967 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:17.910584 1147 connection.cc:573] client connection to 127.0.72.188:33967 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:17.910848 875 connection.cc:573] server connection from 127.0.0.1:54862 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:17.911001 874 connection.cc:573] client connection to 127.0.72.188:33967 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:17.910645 1938 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 14:58:17.612293 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.0.72.130:38257)
1118 14:58:17.669132 (+ 56839us) server_negotiation.cc:207] Beginning negotiation
1118 14:58:17.669140 (+ 8us) server_negotiation.cc:400] Waiting for connection header
1118 14:58:17.669174 (+ 34us) server_negotiation.cc:408] Connection header received
1118 14:58:17.669264 (+ 90us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
1118 14:58:17.669271 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
1118 14:58:17.669362 (+ 91us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
1118 14:58:17.669481 (+ 119us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
1118 14:58:17.910601 (+241120us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49504: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"server-negotiator.queue_time_us":56709,"thread_start_us":58617,"threads_started":1}
W20261118 14:58:17.912153 1012 connection.cc:573] client connection to 127.0.72.188:33967 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:17.944582 589 connection.cc:573] server connection from 127.0.0.1:60894 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
I20261118 14:58:17.945091 290 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu with pid 572
W20261118 14:58:17.963127 1264 connection.cc:573] client connection to 127.0.72.190:42447 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:18.005335 1012 consensus_peers.cc:597] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d -> Peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Couldn't send request to peer c01f1b88968f4905ba5a401a81ade9f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8354c983bc5487e9c94202616d222c6. This is attempt 1: this message will repeat every 5th retry.
W20261118 14:58:18.047061 1010 consensus_peers.cc:597] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d -> Peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Couldn't send request to peer 7c53fe91804746e8bfcf743e0ab8ba6f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cafee7e386434b3c85542d7b6c5024ed. This is attempt 1: this message will repeat every 5th retry.
W20261118 14:58:18.080904 1010 consensus_peers.cc:597] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d -> Peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Couldn't send request to peer 7c53fe91804746e8bfcf743e0ab8ba6f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c8354c983bc5487e9c94202616d222c6. This is attempt 1: this message will repeat every 5th retry.
W20261118 14:58:18.181300 1012 consensus_peers.cc:597] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d -> Peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Couldn't send request to peer c01f1b88968f4905ba5a401a81ade9f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96011b926cca4e4bb2f66fc7944201f7. This is attempt 1: this message will repeat every 5th retry.
W20261118 14:58:18.187206 1012 consensus_peers.cc:597] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d -> Peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Couldn't send request to peer c01f1b88968f4905ba5a401a81ade9f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cafee7e386434b3c85542d7b6c5024ed. This is attempt 1: this message will repeat every 5th retry.
W20261118 14:58:18.238543 1010 consensus_peers.cc:597] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d -> Peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Couldn't send request to peer 7c53fe91804746e8bfcf743e0ab8ba6f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 96011b926cca4e4bb2f66fc7944201f7. This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:18.315044 1967 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:18.315205 1967 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:18.315812 1967 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:18.316294 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a78ecb3a620242f4a16128b0d456c352" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:18.316506 1210 raft_consensus.cc:2490] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
W20261118 14:58:18.316622 875 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:18.316946 874 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d
I20261118 14:58:18.317216 1967 raft_consensus.cc:2826] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:18.317313 1967 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:18.317397 1967 raft_consensus.cc:3096] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:18.319321 1967 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:18.319794 1967 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:18.320192 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a78ecb3a620242f4a16128b0d456c352" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:18.320339 1210 raft_consensus.cc:3096] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
W20261118 14:58:18.320510 875 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:18.322165 1210 raft_consensus.cc:2490] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:18.322515 874 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d
I20261118 14:58:18.322781 1967 raft_consensus.cc:2826] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:18.323120 1967 raft_consensus.cc:705] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:18.323426 1967 consensus_queue.cc:238] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
W20261118 14:58:18.326050 875 consensus_peers.cc:597] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352. This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:18.326411 1972 consensus_queue.cc:1049] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:18.834151 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42447
--webserver_interface=127.0.72.190
--webserver_port=44205
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 14:58:19.172297 1984 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:19.172574 1984 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:19.172657 1984 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:19.181246 1984 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:19.181349 1984 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:19.181411 1984 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:19.181463 1984 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:19.192180 1984 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=44205
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:19.193866 1984 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:19.195722 1984 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:19.203637 1995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:19.203612 1994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:19.205322 1997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:19.205605 1984 server_base.cc:1061] running on GCE node
I20261118 14:58:19.206262 1984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:19.207559 1984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:19.208748 1984 hybrid_clock.cc:648] HybridClock initialized: now 1795013899208695 us; error 48 us; skew 500 ppm
I20261118 14:58:19.209030 1984 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:19.211210 1984 webserver.cc:577] Webserver started at http://127.0.72.190:44205/ using document root <none> and password file <none>
I20261118 14:58:19.211848 1984 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:19.211956 1984 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:19.217227 1984 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20261118 14:58:19.220060 2003 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:19.221412 1984 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20261118 14:58:19.221567 1984 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
uuid: "0ed65a3719634e519d783540afd6189e"
format_stamp: "Formatted at 2026-11-18 14:58:05 on dist-test-slave-qxzn"
I20261118 14:58:19.222174 1984 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:19.238678 1984 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:19.239463 1984 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:19.239715 1984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:19.258060 1984 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42447
I20261118 14:58:19.258082 2055 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42447 every 8 connection(s)
I20261118 14:58:19.259680 1984 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
I20261118 14:58:19.261373 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 1984
I20261118 14:58:19.268087 2056 sys_catalog.cc:263] Verifying existing consensus state
I20261118 14:58:19.272807 2056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Bootstrap starting.
I20261118 14:58:19.295560 2056 log.cc:826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:19.357389 1258 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
I20261118 14:58:19.357519 985 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
I20261118 14:58:19.378477 2056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Bootstrap replayed 1/1 log segments. Stats: ops{read=104 overwritten=0 applied=103 ignored=0} inserts{seen=67 ignored=0} mutations{seen=89 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20261118 14:58:19.379087 2056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Bootstrap complete.
I20261118 14:58:19.384479 2056 raft_consensus.cc:360] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:19.386943 2056 raft_consensus.cc:748] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed65a3719634e519d783540afd6189e, State: Initialized, Role: FOLLOWER
I20261118 14:58:19.387655 2056 consensus_queue.cc:261] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 103, Last appended: 4.104, Last appended by leader: 104, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:19.388383 2068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:19.388615 2068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:19.389349 2056 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:19.402973 2082 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:19.403110 2082 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: loading cluster ID for follower catalog manager: success
I20261118 14:58:19.405263 1123 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
I20261118 14:58:19.408428 2082 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: acquiring CA information for follower catalog manager: success
I20261118 14:58:19.409921 2019 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" instance_seqno: 1795013888201158) as {username='slave'} at 127.0.72.130:39929; Asking this server to re-register.
I20261118 14:58:19.410801 1123 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:19.412053 2082 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:19.412873 2019 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:19.422869 1984 master_runner.cc:187] Error getting master registration for 127.0.72.189:46707: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111),
I20261118 14:58:19.428293 1984 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:19.452626 2086 ts_tablet_manager.cc:979] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Initiating tablet copy from peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:19.453609 2086 tablet_copy_client.cc:323] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: tablet copy: Beginning tablet copy session from remote peer at address 127.0.72.129:41463
I20261118 14:58:19.454628 959 tablet_copy_service.cc:140] P 7c53fe91804746e8bfcf743e0ab8ba6f: Received BeginTabletCopySession request for tablet a78ecb3a620242f4a16128b0d456c352 from peer 245c4fe5e44448d98db1eed6329d150d ({username='slave'} at 127.0.72.130:48619)
I20261118 14:58:19.454805 959 tablet_copy_service.cc:161] P 7c53fe91804746e8bfcf743e0ab8ba6f: Beginning new tablet copy session on tablet a78ecb3a620242f4a16128b0d456c352 from peer 245c4fe5e44448d98db1eed6329d150d at {username='slave'} at 127.0.72.130:48619: session id = 245c4fe5e44448d98db1eed6329d150d-a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:19.457374 959 tablet_copy_source_session.cc:215] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: Tablet Copy: opened 0 blocks and 1 log segments
I20261118 14:58:19.459620 2086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a78ecb3a620242f4a16128b0d456c352. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:19.465170 2086 tablet_copy_client.cc:806] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: tablet copy: Starting download of 0 data blocks...
I20261118 14:58:19.465562 2086 tablet_copy_client.cc:670] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: tablet copy: Starting download of 1 WAL segments...
I20261118 14:58:19.468024 2086 tablet_copy_client.cc:538] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 14:58:19.470747 2086 tablet_bootstrap.cc:492] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:19.475649 2086 tablet_bootstrap.cc:492] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 14:58:19.476022 2086 tablet_bootstrap.cc:492] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap complete.
I20261118 14:58:19.476212 2086 ts_tablet_manager.cc:1449] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20261118 14:58:19.476797 2086 raft_consensus.cc:360] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:19.476953 2086 raft_consensus.cc:748] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:19.477181 2086 consensus_queue.cc:261] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:19.478029 2086 ts_tablet_manager.cc:1480] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20261118 14:58:19.479410 959 tablet_copy_service.cc:342] P 7c53fe91804746e8bfcf743e0ab8ba6f: Request end of tablet copy session 245c4fe5e44448d98db1eed6329d150d-a78ecb3a620242f4a16128b0d456c352 received from {username='slave'} at 127.0.72.130:48619
I20261118 14:58:19.479588 959 tablet_copy_service.cc:434] P 7c53fe91804746e8bfcf743e0ab8ba6f: ending tablet copy session 245c4fe5e44448d98db1eed6329d150d-a78ecb3a620242f4a16128b0d456c352 on tablet a78ecb3a620242f4a16128b0d456c352 with peer 245c4fe5e44448d98db1eed6329d150d
I20261118 14:58:19.797355 2068 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:19.797654 2068 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:19.799652 2068 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:19.800209 2005 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:19.801697 2005 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:19.801905 2006 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:19.802032 2006 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:19.802384 2068 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20261118 14:58:19.862244 2019 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" instance_seqno: 1795013887687621) as {username='slave'} at 127.0.72.129:47005; Asking this server to re-register.
I20261118 14:58:19.863035 985 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:19.864322 2019 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" instance_seqno: 1795013888668865) as {username='slave'} at 127.0.72.131:54735; Asking this server to re-register.
I20261118 14:58:19.864746 2021 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:19.865239 1258 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:19.866695 2021 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:20.240610 1075 raft_consensus.cc:1227] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1] Dedup: 1.1->[]
I20261118 14:58:20.275532 2094 raft_consensus.cc:499] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:58:20.275741 2094 raft_consensus.cc:521] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:20.276484 2094 leader_election.cc:290] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
W20261118 14:58:20.277333 1012 leader_election.cc:343] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Not found: Tablet not found: 309f067161fe469dbb29d14c16139058
W20261118 14:58:20.277503 1010 leader_election.cc:343] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: 309f067161fe469dbb29d14c16139058
I20261118 14:58:20.277674 1010 leader_election.cc:304] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d; no voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7
I20261118 14:58:20.277953 2094 raft_consensus.cc:2771] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:20.421614 2099 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:20.421782 2099 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:20.422545 2099 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:20.424445 2005 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:20.424654 2006 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:20.424786 2006 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:20.425032 2099 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20261118 14:58:21.009773 2107 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:21.009951 2107 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:21.010660 2107 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:21.012403 2005 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:21.012590 2006 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:21.012697 2006 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:21.012919 2107 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20261118 14:58:22.244216 2118 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:22.244442 2118 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:22.245198 2118 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:22.248188 2005 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:22.248430 2006 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:22.248536 2006 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:22.248793 2118 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20261118 14:58:22.768374 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:33967
--webserver_interface=127.0.72.188
--webserver_port=42503
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 14:58:22.846347 2128 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:22.846553 2128 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:22.847241 2128 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:22.849458 2006 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
W20261118 14:58:22.849614 2005 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:22.849802 2005 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:22.850033 2128 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20261118 14:58:23.102697 2126 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:23.103029 2126 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:23.103139 2126 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:23.111937 2126 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:23.112072 2126 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:23.112161 2126 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:23.112227 2126 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:23.123592 2126 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.72.188
--webserver_port=42503
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:23.125451 2126 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:23.127471 2126 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:23.136821 2139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:23.136966 2138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:23.137158 2141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:23.137895 2126 server_base.cc:1061] running on GCE node
I20261118 14:58:23.139159 2126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:23.140844 2126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:23.142089 2126 hybrid_clock.cc:648] HybridClock initialized: now 1795013903142015 us; error 57 us; skew 500 ppm
I20261118 14:58:23.142491 2126 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:23.145016 2126 webserver.cc:577] Webserver started at http://127.0.72.188:42503/ using document root <none> and password file <none>
I20261118 14:58:23.145787 2126 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:23.145891 2126 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:23.152582 2126 fs_manager.cc:714] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20261118 14:58:23.155817 2147 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:23.157603 2126 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20261118 14:58:23.157787 2126 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:23.158653 2126 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:23.196427 2126 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:23.197399 2126 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:23.197711 2126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:23.217073 2126 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.188:33967
I20261118 14:58:23.217073 2199 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.188:33967 every 8 connection(s)
I20261118 14:58:23.218963 2126 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
I20261118 14:58:23.222527 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 2126
I20261118 14:58:23.228013 2200 sys_catalog.cc:263] Verifying existing consensus state
I20261118 14:58:23.232394 2200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap starting.
I20261118 14:58:23.250084 2200 log.cc:826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:23.258777 984 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:23.326299 2200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=104 overwritten=0 applied=103 ignored=0} inserts{seen=67 ignored=0} mutations{seen=89 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20261118 14:58:23.326879 2200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap complete.
I20261118 14:58:23.332482 2200 raft_consensus.cc:360] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:23.334977 2200 raft_consensus.cc:748] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b90720bc0b34e4994abf4fc6f9e59e1, State: Initialized, Role: FOLLOWER
I20261118 14:58:23.335567 2200 consensus_queue.cc:261] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 103, Last appended: 4.104, Last appended by leader: 104, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:23.336344 2207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:23.336606 2207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:23.337225 2200 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:23.350885 2221 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:23.351075 2221 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: loading cluster ID for follower catalog manager: success
I20261118 14:58:23.356077 2221 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: acquiring CA information for follower catalog manager: success
I20261118 14:58:23.367125 1122 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:23.370479 2165 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" instance_seqno: 1795013888201158) as {username='slave'} at 127.0.72.130:49813; Asking this server to re-register.
I20261118 14:58:23.371420 1122 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:23.374486 2165 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:23.379084 2221 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:23.381825 1257 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:23.384403 2164 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" instance_seqno: 1795013888668865) as {username='slave'} at 127.0.72.131:55229; Asking this server to re-register.
I20261118 14:58:23.380187 2126 master_runner.cc:187] Error getting master registration for 127.0.72.189:46707: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111),
I20261118 14:58:23.384963 2126 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:23.385063 1257 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:23.386541 2165 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:23.765187 2165 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" instance_seqno: 1795013887687621) as {username='slave'} at 127.0.72.129:55343; Asking this server to re-register.
I20261118 14:58:23.765796 984 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:23.767113 2165 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:23.835354 2229 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:23.835522 2229 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:23.836277 2229 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:23.838542 2005 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:23.842765 2175 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 5 candidate_status { last_received { term: 4 index: 104 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:23.843289 2175 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed65a3719634e519d783540afd6189e in term 4.
I20261118 14:58:23.843833 2006 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 7c9ef96ff074450a879e66787486386d
I20261118 14:58:23.844102 2229 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20261118 14:58:23.844216 2229 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:23.844310 2229 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 4 FOLLOWER]: Advancing to term 5
I20261118 14:58:23.847544 2229 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:23.848074 2229 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 election: Requested vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:23.848551 2175 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 5 candidate_status { last_received { term: 4 index: 104 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
I20261118 14:58:23.848747 2175 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 4 FOLLOWER]: Advancing to term 5
W20261118 14:58:23.849335 2005 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:23.852032 2175 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed65a3719634e519d783540afd6189e in term 5.
I20261118 14:58:23.852480 2006 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 7c9ef96ff074450a879e66787486386d
I20261118 14:58:23.852807 2229 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 5 FOLLOWER]: Leader election won for term 5
I20261118 14:58:23.853204 2229 raft_consensus.cc:705] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 5 LEADER]: Becoming Leader. State: Replica: 0ed65a3719634e519d783540afd6189e, State: Running, Role: LEADER
I20261118 14:58:23.853662 2229 consensus_queue.cc:238] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 103, Committed index: 103, Last appended: 4.104, Last appended by leader: 104, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:23.856037 2233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ed65a3719634e519d783540afd6189e. Latest consensus state: current_term: 5 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:23.856428 2233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:23.857859 2175 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Refusing update from remote peer 0ed65a3719634e519d783540afd6189e: Log matching property violated. Preceding OpId in replica: term: 4 index: 104. Preceding OpId from leader: term: 5 index: 105. (index mismatch)
W20261118 14:58:23.858377 2005 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e -> Peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Couldn't send request to peer 7c9ef96ff074450a879e66787486386d. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:23.858661 2233 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 105, Last known committed idx: 103, Time since last communication: 0.000s
I20261118 14:58:23.860919 2238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ed65a3719634e519d783540afd6189e. Latest consensus state: current_term: 5 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:23.861178 2238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:23.863273 2233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:23.863485 2233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:23.864403 2237 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 14:58:23.865018 2238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:23.865195 2238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:23.867348 2237 catalog_manager.cc:694] Loaded metadata for table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:23.867894 2237 catalog_manager.cc:694] Loaded metadata for table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:23.868233 2237 catalog_manager.cc:694] Loaded metadata for table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:23.868602 2237 catalog_manager.cc:694] Loaded metadata for table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]
I20261118 14:58:23.869076 2237 catalog_manager.cc:694] Loaded metadata for table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:23.869387 2237 catalog_manager.cc:694] Loaded metadata for table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:23.869838 2237 catalog_manager.cc:694] Loaded metadata for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:23.870169 2237 catalog_manager.cc:694] Loaded metadata for table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]
I20261118 14:58:23.870500 2237 catalog_manager.cc:694] Loaded metadata for table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]
I20261118 14:58:23.870817 2237 catalog_manager.cc:694] Loaded metadata for table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]
I20261118 14:58:23.871271 2237 catalog_manager.cc:694] Loaded metadata for table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]
I20261118 14:58:23.873028 2237 tablet_loader.cc:96] loaded metadata for tablet 03faa53951af43f09a774e20db591133 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b])
I20261118 14:58:23.873656 2237 tablet_loader.cc:96] loaded metadata for tablet 1b5222f6dc7f4757adbf34c3f6a2729e (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28])
I20261118 14:58:23.874222 2237 tablet_loader.cc:96] loaded metadata for tablet 23b3e64e87dd4302a68a8cff81a03da1 (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d])
I20261118 14:58:23.874672 2237 tablet_loader.cc:96] loaded metadata for tablet 42d8a3bab2694c3e83da6a8642d6d911 (table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15])
I20261118 14:58:23.874967 2237 tablet_loader.cc:96] loaded metadata for tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0])
I20261118 14:58:23.875522 2237 tablet_loader.cc:96] loaded metadata for tablet 77e643a760294dacbfca6305f1811bbd (table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8])
I20261118 14:58:23.877245 2237 tablet_loader.cc:96] loaded metadata for tablet a78ecb3a620242f4a16128b0d456c352 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8])
I20261118 14:58:23.878216 2237 tablet_loader.cc:96] loaded metadata for tablet da163bd4e25b4254ab3923af361202ce (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7])
I20261118 14:58:23.878671 2237 tablet_loader.cc:96] loaded metadata for tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6])
I20261118 14:58:23.878948 2237 tablet_loader.cc:96] loaded metadata for tablet e47d7d10a5c741ecb30ec701c76b27a3 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f])
I20261118 14:58:23.879518 2237 tablet_loader.cc:96] loaded metadata for tablet f87ee223b5a246c58e9b2eca5740d08c (table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627])
I20261118 14:58:23.879760 2237 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:23.880821 2237 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:23.880899 2237 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:23.882566 2237 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:23.883657 2237 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Loaded TSK: 0
I20261118 14:58:23.884460 2237 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:23.885764 2237 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:23.887082 2237 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 33
I20261118 14:58:23.895645 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet 96011b926cca4e4bb2f66fc7944201f7 with delete_type TABLET_DATA_DELETED (replaced by tablet a78ecb3a620242f4a16128b0d456c352) from {username='slave'} at 127.0.0.1:58928
I20261118 14:58:23.896204 2246 tablet_replica.cc:333] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:23.896389 2246 raft_consensus.cc:2261] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:23.896742 2246 raft_consensus.cc:2290] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:23.897214 2246 ts_tablet_manager.cc:1962] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:23.901535 2246 ts_tablet_manager.cc:1975] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:23.901630 2246 log.cc:1199] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/96011b926cca4e4bb2f66fc7944201f7
I20261118 14:58:23.901954 2246 ts_tablet_manager.cc:1996] T 96011b926cca4e4bb2f66fc7944201f7 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:23.903004 2007 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 96011b926cca4e4bb2f66fc7944201f7 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]) successfully deleted
I20261118 14:58:23.923867 2019 master_service.cc:965] ReplaceTablet: received request to replace tablet cafee7e386434b3c85542d7b6c5024ed from {username='slave'} at 127.0.0.1:37218
I20261118 14:58:23.924021 2018 master_service.cc:965] ReplaceTablet: received request to replace tablet 522efb108dd140b8a3fd476d202a073e from {username='slave'} at 127.0.0.1:37218
I20261118 14:58:23.923838 2021 master_service.cc:965] ReplaceTablet: received request to replace tablet 03faa53951af43f09a774e20db591133 from {username='slave'} at 127.0.0.1:37218
I20261118 14:58:23.924767 2016 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37218:
table { table_name: "default.Table_a0ea13e08f4e45bc84d5936c615c8ff7" } new_table_name: "default.Table_8afe8b6b4fe4433882ded7831ce722e9" modify_external_catalogs: true
I20261118 14:58:23.925182 2017 master_service.cc:965] ReplaceTablet: received request to replace tablet 309f067161fe469dbb29d14c16139058 from {username='slave'} at 127.0.0.1:37218
I20261118 14:58:23.925800 2019 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37218:
table { table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" } modify_external_catalogs: true
I20261118 14:58:23.926021 2019 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37218:
table { table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" } modify_external_catalogs: true
I20261118 14:58:23.925926 2015 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37218:
name: "default.Table_0cbce4af337d4a5baeb313a32662d446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:23.925761 2012 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37218:
table { table_name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1" } modify_external_catalogs: true
I20261118 14:58:23.927503 2012 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37218:
table { table_name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1" } modify_external_catalogs: true
I20261118 14:58:23.927843 2017 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37218:
name: "default.Table_5c2ab051f54546cab77c75d60b2a8bd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:23.927928 2014 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37218:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:23.928217 2014 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37218:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:23.928586 2013 master_service.cc:965] ReplaceTablet: received request to replace tablet f66c4e0f06f043108a6451666a80a5f9 from {username='slave'} at 127.0.0.1:37218
I20261118 14:58:23.928522 2011 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37218:
name: "default.Table_6c65bee0620c494591ca0d8d3226c6a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:23.929926 2013 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37218:
name: "default.Table_36b9017c9ae24ecdb8ae81bf85341052"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20261118 14:58:23.932314 2081 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 36, previous 36; dumping first 4 out of 5 received events: NotificationEvent(eventId=34, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_4789eba8c66842f6ae326364062589fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_4789eba8c66842f6ae326364062589fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4789eba8c66842f6ae326364062589fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4789eba8c66842f6ae326364062589fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4789eba8c66842f6ae326364062589fb\",\"kudu.table_id\":\"0ce3ba58cd7440a9b27427e7713af331\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=35, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_7171b2f0b6ea470393bf9aa28246cf65, message={"server":"","servicePrincipal":"","db":"default","table":"table_7171b2f0b6ea470393bf9aa28246cf65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_7171b2f0b6ea470393bf9aa28246cf65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_7171b2f0b6ea470393bf9aa28246cf65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_7171b2f0b6ea470393bf9aa28246cf65\",\"kudu.table_id\":\"cdc2cda186804cd39b0d320f08e91df8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_4d2c8b2bcb994782a8bbf4b38ca2f7af, message={"server":"","servicePrincipal":"","db":"default","table":"table_4d2c8b2bcb994782a8bbf4b38ca2f7af","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af\",\"kudu.table_id\":\"d65d9ba97cda44e3bdff3add1fec1ec8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_36b9017c9ae24ecdb8ae81bf85341052, message={"server":"","servicePrincipal":"","db":"default","table":"table_36b9017c9ae24ecdb8ae81bf85341052","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_36b9017c9ae24ecdb8ae81bf85341052\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_36b9017c9ae24ecdb8ae81bf85341052\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_36b9017c9ae24ecdb8ae81bf85341052\",\"kudu.table_id\":\"378bd168f216477992c1b7bb33791778\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20261118 14:58:23.935577 2021 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Sending DeleteTablet for 3 replicas of tablet 03faa53951af43f09a774e20db591133
@ 0x7efef7fa6d5d google::LogMessage::Fail() at ??:0
I20261118 14:58:23.938938 2018 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Sending DeleteTablet for 3 replicas of tablet 522efb108dd140b8a3fd476d202a073e
I20261118 14:58:23.939724 2018 catalog_manager.cc:6936] ReplaceTablet: tablet 522efb108dd140b8a3fd476d202a073e deleted and replaced by tablet 6ffda5e2164e4a558231db3043422159
I20261118 14:58:23.941259 2021 catalog_manager.cc:6936] ReplaceTablet: tablet 03faa53951af43f09a774e20db591133 deleted and replaced by tablet d5ebf8d258a644d9bea96b42357a9345
I20261118 14:58:23.942641 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet 522efb108dd140b8a3fd476d202a073e with delete_type TABLET_DATA_DELETED (replaced by tablet 6ffda5e2164e4a558231db3043422159) from {username='slave'} at 127.0.0.1:58928
I20261118 14:58:23.942934 2246 tablet_replica.cc:333] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:23.943411 2246 raft_consensus.cc:2261] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:23.943562 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet 03faa53951af43f09a774e20db591133 with delete_type TABLET_DATA_DELETED (replaced by tablet d5ebf8d258a644d9bea96b42357a9345) from {username='slave'} at 127.0.0.1:58928
I20261118 14:58:23.943759 2246 raft_consensus.cc:2290] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:23.944588 2246 ts_tablet_manager.cc:1962] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7efef7faab23 google::LogMessage::SendToLog() at ??:0
@ 0x7efef7fa675c google::LogMessage::Flush() at ??:0
@ 0x7efef7fa69a1 google::LogMessage::~LogMessage() at ??:0
I20261118 14:58:23.950994 2246 ts_tablet_manager.cc:1975] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:23.951108 2246 log.cc:1199] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/522efb108dd140b8a3fd476d202a073e
I20261118 14:58:23.953496 2246 ts_tablet_manager.cc:1996] T 522efb108dd140b8a3fd476d202a073e P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:23.955144 2007 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 522efb108dd140b8a3fd476d202a073e (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]) successfully deleted
I20261118 14:58:23.955452 2246 tablet_replica.cc:333] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:23.955617 2246 raft_consensus.cc:2261] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:23.955796 2246 raft_consensus.cc:2290] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:23.956301 2246 ts_tablet_manager.cc:1962] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:23.957607 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet 03faa53951af43f09a774e20db591133 with delete_type TABLET_DATA_DELETED (replaced by tablet d5ebf8d258a644d9bea96b42357a9345) from {username='slave'} at 127.0.0.1:36562
I20261118 14:58:23.957530 2021 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37218:
name: "default.Table_d0b14f4adfe3441c88447b2fc1e2d506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:23.957930 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet 522efb108dd140b8a3fd476d202a073e with delete_type TABLET_DATA_DELETED (replaced by tablet 6ffda5e2164e4a558231db3043422159) from {username='slave'} at 127.0.0.1:36562
I20261118 14:58:23.959640 2018 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37218:
name: "default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:23.960019 2252 tablet_replica.cc:333] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:23.960217 2252 raft_consensus.cc:2261] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:23.960564 2252 raft_consensus.cc:2290] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:23.960534 915 tablet_service.cc:1547] Processing DeleteTablet for tablet 522efb108dd140b8a3fd476d202a073e with delete_type TABLET_DATA_DELETED (replaced by tablet 6ffda5e2164e4a558231db3043422159) from {username='slave'} at 127.0.0.1:58670
I20261118 14:58:23.960641 919 tablet_service.cc:1547] Processing DeleteTablet for tablet 03faa53951af43f09a774e20db591133 with delete_type TABLET_DATA_DELETED (replaced by tablet d5ebf8d258a644d9bea96b42357a9345) from {username='slave'} at 127.0.0.1:58670
I20261118 14:58:23.961134 2252 ts_tablet_manager.cc:1962] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7eff0d4269a4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 14:58:23.965982 2246 ts_tablet_manager.cc:1975] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:23.966081 2246 log.cc:1199] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/03faa53951af43f09a774e20db591133
I20261118 14:58:23.966499 2246 ts_tablet_manager.cc:1996] T 03faa53951af43f09a774e20db591133 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:23.967521 2007 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 03faa53951af43f09a774e20db591133 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
@ 0x7eff0d425251 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7eff0d42cecc kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 14:58:23.972872 2252 ts_tablet_manager.cc:1975] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:23.973003 2252 log.cc:1199] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/03faa53951af43f09a774e20db591133
I20261118 14:58:23.973402 2252 ts_tablet_manager.cc:1996] T 03faa53951af43f09a774e20db591133 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:23.973546 2253 tablet_replica.cc:333] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:23.973732 2253 raft_consensus.cc:2261] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:23.973941 2253 raft_consensus.cc:2290] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:23.974319 2252 tablet_replica.cc:333] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:23.974514 2253 ts_tablet_manager.cc:1962] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:23.974490 2006 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet 03faa53951af43f09a774e20db591133 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]) successfully deleted
I20261118 14:58:23.974532 2252 raft_consensus.cc:2261] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:23.974677 2252 raft_consensus.cc:2290] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7eff0d42cd49 std::_Function_handler<>::_M_invoke() at ??:0
I20261118 14:58:23.975179 2252 ts_tablet_manager.cc:1962] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x372c7d std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
@ 0x7efef93efdc9 kudu::Thread::SuperviseThread() at ??:0
I20261118 14:58:23.980815 2253 ts_tablet_manager.cc:1975] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:23.980923 2253 log.cc:1199] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/522efb108dd140b8a3fd476d202a073e
I20261118 14:58:23.981024 2252 ts_tablet_manager.cc:1975] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:23.981127 2252 log.cc:1199] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/522efb108dd140b8a3fd476d202a073e
@ 0x7efefcaca6db start_thread at ??:0
I20261118 14:58:23.981309 2253 ts_tablet_manager.cc:1996] T 522efb108dd140b8a3fd476d202a073e P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:23.981452 2252 ts_tablet_manager.cc:1996] T 522efb108dd140b8a3fd476d202a073e P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
@ 0x7efef553a61f clone at ??:0
I20261118 14:58:23.982213 2253 tablet_replica.cc:333] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:23.982635 2253 raft_consensus.cc:2261] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:23.982784 2253 raft_consensus.cc:2290] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:23.983275 2253 ts_tablet_manager.cc:1962] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:23.989077 2253 ts_tablet_manager.cc:1975] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:23.989189 2253 log.cc:1199] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/03faa53951af43f09a774e20db591133
I20261118 14:58:23.989527 2253 ts_tablet_manager.cc:1996] T 03faa53951af43f09a774e20db591133 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:24.054692 1123 heartbeater.cc:500] Master 127.0.72.190:42447 was elected leader, sending a full tablet report...
W20261118 14:58:24.154349 874 connection.cc:573] server connection from 127.0.0.1:58670 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:24.154340 1148 connection.cc:573] server connection from 127.0.0.1:36562 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:24.155694 1010 connection.cc:573] client connection to 127.0.72.190:42447 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:24.156138 1264 connection.cc:573] client connection to 127.0.72.190:42447 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:24.157065 1264 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.190:42447)
W20261118 14:58:24.157666 1264 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.190:42447)
W20261118 14:58:24.157850 1264 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.190:42447)
W20261118 14:58:24.158144 1264 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.190:42447)
I20261118 14:58:24.386222 2095 raft_consensus.cc:499] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:58:24.386441 2095 raft_consensus.cc:521] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:24.387171 2095 leader_election.cc:290] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
W20261118 14:58:24.387912 1012 leader_election.cc:343] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Not found: Tablet not found: 309f067161fe469dbb29d14c16139058
W20261118 14:58:24.388083 1010 leader_election.cc:343] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: 309f067161fe469dbb29d14c16139058
I20261118 14:58:24.388177 1010 leader_election.cc:304] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d; no voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7
I20261118 14:58:24.388386 2095 raft_consensus.cc:2771] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:24.472301 2268 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:24.472494 2268 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:24.474277 2268 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:24.475051 2148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:24.476307 2148 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:24.476526 2149 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:24.476647 2149 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:24.476991 2268 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 14:58:25.205325 2280 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:25.205504 2280 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:25.206185 2280 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:25.208135 2149 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:25.208293 2148 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:25.208408 2148 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:25.208691 2280 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 14:58:26.247728 2295 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:26.247907 2295 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:26.248445 2295 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:26.250164 2149 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:26.250376 2148 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:26.250490 2148 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:26.250705 2295 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 14:58:26.434067 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42447
--webserver_interface=127.0.72.190
--webserver_port=44205
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 14:58:26.767668 2303 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:26.768014 2303 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:26.768131 2303 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:26.776883 2303 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:26.776970 2303 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:26.777035 2303 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:26.777100 2303 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:26.788089 2303 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=44205
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:26.789731 2303 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:26.791496 2303 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:26.799327 2312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:26.799300 2313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:26.799983 2315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:26.800400 2303 server_base.cc:1061] running on GCE node
I20261118 14:58:26.801059 2303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:26.802393 2303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:26.803599 2303 hybrid_clock.cc:648] HybridClock initialized: now 1795013906803545 us; error 46 us; skew 500 ppm
I20261118 14:58:26.803910 2303 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:26.805883 2303 webserver.cc:577] Webserver started at http://127.0.72.190:44205/ using document root <none> and password file <none>
I20261118 14:58:26.806581 2303 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:26.806684 2303 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:26.811702 2303 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20261118 14:58:26.814390 2321 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:26.815758 2303 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20261118 14:58:26.815922 2303 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
uuid: "0ed65a3719634e519d783540afd6189e"
format_stamp: "Formatted at 2026-11-18 14:58:05 on dist-test-slave-qxzn"
I20261118 14:58:26.816557 2303 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:26.867309 2303 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:26.868232 2303 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:26.868498 2303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:26.889719 2303 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42447
I20261118 14:58:26.889719 2374 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42447 every 8 connection(s)
I20261118 14:58:26.891770 2303 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-0/data/info.pb
I20261118 14:58:26.898581 2375 sys_catalog.cc:263] Verifying existing consensus state
I20261118 14:58:26.901614 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 2303
I20261118 14:58:26.901958 2375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Bootstrap starting.
I20261118 14:58:26.922786 2375 log.cc:826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:27.001667 2375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Bootstrap replayed 1/1 log segments. Stats: ops{read=108 overwritten=0 applied=108 ignored=0} inserts{seen=69 ignored=0} mutations{seen=96 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 14:58:27.002345 2375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Bootstrap complete.
I20261118 14:58:27.007591 2375 raft_consensus.cc:360] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:27.008219 2375 raft_consensus.cc:748] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed65a3719634e519d783540afd6189e, State: Initialized, Role: FOLLOWER
I20261118 14:58:27.008873 2375 consensus_queue.cc:261] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 108, Last appended: 5.108, Last appended by leader: 108, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:27.009601 2379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:27.009832 2379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:27.010620 2375 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:27.021476 2393 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:27.021646 2393 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: loading cluster ID for follower catalog manager: success
I20261118 14:58:27.027973 2393 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: acquiring CA information for follower catalog manager: success
I20261118 14:58:27.036136 2303 master_runner.cc:187] Error getting master registration for 127.0.72.189:46707: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111),
I20261118 14:58:27.041952 2303 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:27.042529 2393 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:27.126941 2397 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:27.127144 2397 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:27.127803 2397 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:27.130468 2149 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:27.135815 2348 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" candidate_term: 6 candidate_status { last_received { term: 5 index: 108 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e" is_pre_election: true
I20261118 14:58:27.136386 2348 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b90720bc0b34e4994abf4fc6f9e59e1 in term 5.
I20261118 14:58:27.136981 2148 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 7c9ef96ff074450a879e66787486386d
I20261118 14:58:27.137307 2397 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20261118 14:58:27.137439 2397 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:27.137558 2397 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 5 FOLLOWER]: Advancing to term 6
I20261118 14:58:27.140923 2397 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:27.141748 2397 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 election: Requested vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
I20261118 14:58:27.142396 2348 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" candidate_term: 6 candidate_status { last_received { term: 5 index: 108 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e"
I20261118 14:58:27.142637 2348 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 5 FOLLOWER]: Advancing to term 6
W20261118 14:58:27.143519 2149 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:27.147162 2348 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b90720bc0b34e4994abf4fc6f9e59e1 in term 6.
I20261118 14:58:27.147635 2148 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 7c9ef96ff074450a879e66787486386d
I20261118 14:58:27.147928 2397 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 6 FOLLOWER]: Leader election won for term 6
I20261118 14:58:27.148403 2397 raft_consensus.cc:705] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 6 LEADER]: Becoming Leader. State: Replica: 9b90720bc0b34e4994abf4fc6f9e59e1, State: Running, Role: LEADER
I20261118 14:58:27.148962 2397 consensus_queue.cc:238] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 108, Committed index: 108, Last appended: 5.108, Last appended by leader: 108, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:27.151736 2402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b90720bc0b34e4994abf4fc6f9e59e1. Latest consensus state: current_term: 6 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:27.152168 2402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:27.154492 2348 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Refusing update from remote peer 9b90720bc0b34e4994abf4fc6f9e59e1: Log matching property violated. Preceding OpId in replica: term: 5 index: 108. Preceding OpId from leader: term: 6 index: 109. (index mismatch)
I20261118 14:58:27.156067 2406 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 14:58:27.156332 2402 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 109, Last known committed idx: 108, Time since last communication: 0.000s
W20261118 14:58:27.157059 2149 consensus_peers.cc:597] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 -> Peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Couldn't send request to peer 7c9ef96ff074450a879e66787486386d. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:27.160770 2406 catalog_manager.cc:694] Loaded metadata for table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:27.161192 2379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b90720bc0b34e4994abf4fc6f9e59e1. Latest consensus state: current_term: 6 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:27.161471 2379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:27.161640 2406 catalog_manager.cc:694] Loaded metadata for table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:27.162092 2406 catalog_manager.cc:694] Loaded metadata for table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:27.162585 2406 catalog_manager.cc:694] Loaded metadata for table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]
I20261118 14:58:27.163285 2406 catalog_manager.cc:694] Loaded metadata for table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:27.163751 2406 catalog_manager.cc:694] Loaded metadata for table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:27.164543 2406 catalog_manager.cc:694] Loaded metadata for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:27.165060 2406 catalog_manager.cc:694] Loaded metadata for table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]
I20261118 14:58:27.165498 2406 catalog_manager.cc:694] Loaded metadata for table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]
I20261118 14:58:27.165984 2406 catalog_manager.cc:694] Loaded metadata for table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]
I20261118 14:58:27.166718 2406 catalog_manager.cc:694] Loaded metadata for table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]
I20261118 14:58:27.170703 2406 tablet_loader.cc:96] loaded metadata for tablet 1b5222f6dc7f4757adbf34c3f6a2729e (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28])
I20261118 14:58:27.171600 2406 tablet_loader.cc:96] loaded metadata for tablet 23b3e64e87dd4302a68a8cff81a03da1 (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d])
I20261118 14:58:27.172259 2406 tablet_loader.cc:96] loaded metadata for tablet 42d8a3bab2694c3e83da6a8642d6d911 (table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15])
I20261118 14:58:27.173110 2406 tablet_loader.cc:96] loaded metadata for tablet 6ffda5e2164e4a558231db3043422159 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0])
I20261118 14:58:27.173967 2406 tablet_loader.cc:96] loaded metadata for tablet 77e643a760294dacbfca6305f1811bbd (table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8])
I20261118 14:58:27.176445 2406 tablet_loader.cc:96] loaded metadata for tablet a78ecb3a620242f4a16128b0d456c352 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8])
I20261118 14:58:27.178059 2406 tablet_loader.cc:96] loaded metadata for tablet d5ebf8d258a644d9bea96b42357a9345 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b])
I20261118 14:58:27.178506 2406 tablet_loader.cc:96] loaded metadata for tablet da163bd4e25b4254ab3923af361202ce (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7])
I20261118 14:58:27.179162 2406 tablet_loader.cc:96] loaded metadata for tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6])
I20261118 14:58:27.179579 2406 tablet_loader.cc:96] loaded metadata for tablet e47d7d10a5c741ecb30ec701c76b27a3 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f])
I20261118 14:58:27.180594 2406 tablet_loader.cc:96] loaded metadata for tablet f87ee223b5a246c58e9b2eca5740d08c (table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627])
I20261118 14:58:27.181030 2406 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:27.182986 2406 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:27.183111 2406 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:27.184680 1123 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
I20261118 14:58:27.184891 1123 heartbeater.cc:500] Master 127.0.72.190:42447 was elected leader, sending a full tablet report...
I20261118 14:58:27.185361 2406 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:27.187047 2406 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Loaded TSK: 0
I20261118 14:58:27.188440 2406 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:27.190819 2406 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:27.192735 2406 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 33
I20261118 14:58:27.196408 2379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:27.196622 2379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:27.197808 2338 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" instance_seqno: 1795013888201158) as {username='slave'} at 127.0.72.130:46083; Asking this server to re-register.
I20261118 14:58:27.199282 1123 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:27.203886 2338 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:27.204689 2397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:27.205281 2397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:27.210206 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet c8354c983bc5487e9c94202616d222c6 with delete_type TABLET_DATA_DELETED (replaced by tablet da163bd4e25b4254ab3923af361202ce) from {username='slave'} at 127.0.0.1:58934
I20261118 14:58:27.210901 2412 tablet_replica.cc:333] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:27.211125 2412 raft_consensus.cc:2261] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:27.211550 2412 raft_consensus.cc:2290] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:27.212239 2412 ts_tablet_manager.cc:1962] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:27.219884 2412 ts_tablet_manager.cc:1975] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:27.220038 2412 log.cc:1199] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/c8354c983bc5487e9c94202616d222c6
I20261118 14:58:27.220508 2412 ts_tablet_manager.cc:1996] T c8354c983bc5487e9c94202616d222c6 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:27.221942 2151 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet c8354c983bc5487e9c94202616d222c6 (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]) successfully deleted
I20261118 14:58:27.262933 2165 master_service.cc:965] ReplaceTablet: received request to replace tablet 1b5222f6dc7f4757adbf34c3f6a2729e from {username='slave'} at 127.0.0.1:35344
I20261118 14:58:27.264683 2163 master_service.cc:965] ReplaceTablet: received request to replace tablet 309f067161fe469dbb29d14c16139058 from {username='slave'} at 127.0.0.1:35344
I20261118 14:58:27.264887 2162 master_service.cc:965] ReplaceTablet: received request to replace tablet e47d7d10a5c741ecb30ec701c76b27a3 from {username='slave'} at 127.0.0.1:35344
I20261118 14:58:27.265482 2164 master_service.cc:965] ReplaceTablet: received request to replace tablet 23b3e64e87dd4302a68a8cff81a03da1 from {username='slave'} at 127.0.0.1:35344
I20261118 14:58:27.265708 2161 master_service.cc:965] ReplaceTablet: received request to replace tablet a78ecb3a620242f4a16128b0d456c352 from {username='slave'} at 127.0.0.1:35344
I20261118 14:58:27.266769 2160 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35344:
table { table_name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9" } modify_external_catalogs: true
I20261118 14:58:27.267022 2160 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35344:
table { table_name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9" } modify_external_catalogs: true
I20261118 14:58:27.264137 2157 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35344:
name: "default.Table_d24daba84f8c4c94824da319564f0453"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:27.267274 2159 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35344:
name: "default.Table_4dba69207cca45529e828162200c13f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:27.265856 2158 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35344:
name: "default.Table_053874e1edfe4b8eac52b4ba5f1b1870"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:27.283290 2163 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35344:
table { table_name: "default.Table_7171b2f0b6ea470393bf9aa28246cf65" } new_table_name: "default.Table_de4cb9add02543cb98fda40cb17f0587" modify_external_catalogs: true
F20261118 14:58:27.284612 2220 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 36, previous 36; dumping first 4 out of 5 received events: NotificationEvent(eventId=34, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_4789eba8c66842f6ae326364062589fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_4789eba8c66842f6ae326364062589fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4789eba8c66842f6ae326364062589fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4789eba8c66842f6ae326364062589fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4789eba8c66842f6ae326364062589fb\",\"kudu.table_id\":\"0ce3ba58cd7440a9b27427e7713af331\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=35, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_7171b2f0b6ea470393bf9aa28246cf65, message={"server":"","servicePrincipal":"","db":"default","table":"table_7171b2f0b6ea470393bf9aa28246cf65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_7171b2f0b6ea470393bf9aa28246cf65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_7171b2f0b6ea470393bf9aa28246cf65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_7171b2f0b6ea470393bf9aa28246cf65\",\"kudu.table_id\":\"cdc2cda186804cd39b0d320f08e91df8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_4d2c8b2bcb994782a8bbf4b38ca2f7af, message={"server":"","servicePrincipal":"","db":"default","table":"table_4d2c8b2bcb994782a8bbf4b38ca2f7af","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af\",\"kudu.table_id\":\"d65d9ba97cda44e3bdff3add1fec1ec8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_36b9017c9ae24ecdb8ae81bf85341052, message={"server":"","servicePrincipal":"","db":"default","table":"table_36b9017c9ae24ecdb8ae81bf85341052","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_36b9017c9ae24ecdb8ae81bf85341052\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_36b9017c9ae24ecdb8ae81bf85341052\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_36b9017c9ae24ecdb8ae81bf85341052\",\"kudu.table_id\":\"378bd168f216477992c1b7bb33791778\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20261118 14:58:27.266283 2156 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35344:
name: "default.Table_4789eba8c66842f6ae326364062589fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f6dcd2cad5d google::LogMessage::Fail() at ??:0
I20261118 14:58:27.290274 985 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
@ 0x7f6dcd2ceb23 google::LogMessage::SendToLog() at ??:0
I20261118 14:58:27.291735 2338 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" instance_seqno: 1795013887687621) as {username='slave'} at 127.0.72.129:56509; Asking this server to re-register.
I20261118 14:58:27.292383 985 heartbeater.cc:462] Registering TS with master...
@ 0x7f6dcd2ca75c google::LogMessage::Flush() at ??:0
I20261118 14:58:27.293735 2338 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
@ 0x7f6dcd2ca9a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f6de274a9a4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f6de2749251 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
@ 0x7f6de2750ecc kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7f6de2750d49 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x372c7d std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
@ 0x7f6dce713dc9 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6dd1dee6db start_thread at ??:0
@ 0x7f6dca85e61f clone at ??:0
I20261118 14:58:27.410130 1258 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42447
I20261118 14:58:27.411643 2338 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" instance_seqno: 1795013888668865) as {username='slave'} at 127.0.72.131:33941; Asking this server to re-register.
I20261118 14:58:27.412258 1258 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:27.413847 2338 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:27.437060 1122 heartbeater.cc:500] Master 127.0.72.188:33967 was elected leader, sending a full tablet report...
W20261118 14:58:27.503645 1147 connection.cc:573] client connection to 127.0.72.188:33967 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:27.503749 2323 connection.cc:573] server connection from 127.0.0.1:37754 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:27.504273 1012 connection.cc:573] client connection to 127.0.72.188:33967 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:27.504320 1266 connection.cc:573] client connection to 127.0.72.188:33967 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:27.505028 1266 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.188:33967)
W20261118 14:58:27.505272 1266 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.188:33967)
W20261118 14:58:27.505492 1266 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.188:33967)
W20261118 14:58:27.505925 1266 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.188:33967)
W20261118 14:58:27.676555 1010 consensus_peers.cc:597] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d -> Peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Couldn't send request to peer 7c53fe91804746e8bfcf743e0ab8ba6f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cafee7e386434b3c85542d7b6c5024ed. This is attempt 6: this message will repeat every 5th retry.
I20261118 14:58:27.799485 2437 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:27.799696 2437 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:27.801330 2437 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:27.802130 2323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:27.803340 2324 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
W20261118 14:58:27.803507 2323 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:27.803687 2323 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:27.804013 2437 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 14:58:28.313313 2443 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:28.313479 2443 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:28.314083 2443 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:28.316030 2323 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:28.316212 2324 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:28.316326 2324 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:28.316532 2443 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 14:58:29.014259 2454 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:29.014483 2454 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:29.015074 2454 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:29.016866 2323 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:29.017002 2324 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:29.017141 2324 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:29.017359 2454 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20261118 14:58:29.641408 1012 consensus_peers.cc:597] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d -> Peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Couldn't send request to peer c01f1b88968f4905ba5a401a81ade9f7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cafee7e386434b3c85542d7b6c5024ed. This is attempt 6: this message will repeat every 5th retry.
I20261118 14:58:29.694975 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:46707
--webserver_interface=127.0.72.189
--webserver_port=42561
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 14:58:29.728711 2465 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:29.728890 2465 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:29.729542 2465 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:29.731451 2323 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:29.732079 2324 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:29.732200 2324 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ed65a3719634e519d783540afd6189e; no voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:29.732429 2465 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20261118 14:58:30.022073 2464 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:30.022406 2464 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:30.022518 2464 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:30.030879 2464 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:30.030966 2464 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:30.031051 2464 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:30.031113 2464 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:30.041005 2464 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:46707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=42561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:30.042570 2464 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:30.044277 2464 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:30.051766 2473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:30.052155 2474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:30.052649 2476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:30.052709 2464 server_base.cc:1061] running on GCE node
I20261118 14:58:30.053450 2464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:30.054807 2464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:30.056001 2464 hybrid_clock.cc:648] HybridClock initialized: now 1795013910055948 us; error 44 us; skew 500 ppm
I20261118 14:58:30.056288 2464 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:30.058287 2464 webserver.cc:577] Webserver started at http://127.0.72.189:42561/ using document root <none> and password file <none>
I20261118 14:58:30.058955 2464 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:30.059054 2464 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:30.064040 2464 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20261118 14:58:30.066672 2482 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:30.067948 2464 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20261118 14:58:30.068090 2464 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
uuid: "7c9ef96ff074450a879e66787486386d"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:30.068703 2464 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:30.094533 2464 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:30.095258 2464 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:30.095501 2464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:30.113363 2464 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:46707
I20261118 14:58:30.113384 2533 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:46707 every 8 connection(s)
I20261118 14:58:30.114949 2464 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
I20261118 14:58:30.119807 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 2464
I20261118 14:58:30.122264 2534 sys_catalog.cc:263] Verifying existing consensus state
I20261118 14:58:30.125918 2534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap starting.
I20261118 14:58:30.141670 2534 log.cc:826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:30.208056 2534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap replayed 1/1 log segments. Stats: ops{read=97 overwritten=0 applied=97 ignored=0} inserts{seen=63 ignored=0} mutations{seen=84 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 14:58:30.208559 2534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap complete.
I20261118 14:58:30.214919 2534 raft_consensus.cc:360] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:30.215796 2534 raft_consensus.cc:748] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9ef96ff074450a879e66787486386d, State: Initialized, Role: FOLLOWER
I20261118 14:58:30.216539 2534 consensus_queue.cc:261] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 97, Last appended: 3.97, Last appended by leader: 97, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:30.217324 2539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:30.217612 2539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:30.218652 2534 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:30.241792 2553 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:30.241901 2553 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: loading cluster ID for follower catalog manager: success
I20261118 14:58:30.246155 2553 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: acquiring CA information for follower catalog manager: success
I20261118 14:58:30.248844 2553 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:30.256019 2464 master_runner.cc:187] Error getting master registration for 127.0.72.188:33967: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111),
I20261118 14:58:30.261539 2464 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:30.299697 986 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:30.303170 2499 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" instance_seqno: 1795013887687621) as {username='slave'} at 127.0.72.129:47557; Asking this server to re-register.
I20261118 14:58:30.303970 986 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:30.305761 2499 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:30.415364 1259 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:30.417446 2499 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" instance_seqno: 1795013888668865) as {username='slave'} at 127.0.72.131:48237; Asking this server to re-register.
I20261118 14:58:30.418185 1259 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:30.419744 2499 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:30.439591 1124 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:30.441164 2499 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" instance_seqno: 1795013888201158) as {username='slave'} at 127.0.72.130:44267; Asking this server to re-register.
I20261118 14:58:30.441742 1124 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:30.442950 2499 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:30.668593 2560 raft_consensus.cc:499] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:58:30.668797 2560 raft_consensus.cc:521] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:30.669682 2560 leader_election.cc:290] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
W20261118 14:58:30.670688 1012 leader_election.cc:343] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): Not found: Tablet not found: 309f067161fe469dbb29d14c16139058
W20261118 14:58:30.670883 1010 leader_election.cc:343] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: 309f067161fe469dbb29d14c16139058
I20261118 14:58:30.670992 1010 leader_election.cc:304] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d; no voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7
I20261118 14:58:30.671288 2560 raft_consensus.cc:2771] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:30.736423 2564 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:30.736649 2564 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:30.738435 2564 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:30.738921 2348 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 4 candidate_status { last_received { term: 3 index: 97 } } ignore_live_leader: false dest_uuid: "0ed65a3719634e519d783540afd6189e" is_pre_election: true
W20261118 14:58:30.739064 2485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:30.739204 2348 raft_consensus.cc:2390] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c9ef96ff074450a879e66787486386d for earlier term 4. Current term is 6.
I20261118 14:58:30.739873 2483 leader_election.cc:400] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 4 pre-election: Vote denied by peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c9ef96ff074450a879e66787486386d for earlier term 4. Current term is 6.
I20261118 14:58:30.739952 2483 leader_election.cc:403] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20261118 14:58:30.740234 2564 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 3 FOLLOWER]: Advancing to term 6
W20261118 14:58:30.740470 2485 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:30.743767 2564 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 6 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c9ef96ff074450a879e66787486386d for earlier term 4. Current term is 6.
I20261118 14:58:31.163897 2569 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:31.164098 2569 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:31.164736 2569 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:31.167800 2324 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:31.172933 2509 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 7 candidate_status { last_received { term: 6 index: 112 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d" is_pre_election: true
I20261118 14:58:31.173393 2509 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed65a3719634e519d783540afd6189e in term 6.
I20261118 14:58:31.173974 2323 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d; no voters: 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:31.174270 2569 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20261118 14:58:31.174418 2569 raft_consensus.cc:499] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9b90720bc0b34e4994abf4fc6f9e59e1)
I20261118 14:58:31.174531 2569 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 6 FOLLOWER]: Advancing to term 7
I20261118 14:58:31.176525 2569 raft_consensus.cc:521] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:31.177039 2569 leader_election.cc:290] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 election: Requested vote from peers 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:31.177455 2509 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ed65a3719634e519d783540afd6189e" candidate_term: 7 candidate_status { last_received { term: 6 index: 112 } } ignore_live_leader: false dest_uuid: "7c9ef96ff074450a879e66787486386d"
I20261118 14:58:31.177630 2509 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 6 FOLLOWER]: Advancing to term 7
W20261118 14:58:31.178328 2324 leader_election.cc:336] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:31.179913 2509 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed65a3719634e519d783540afd6189e in term 7.
I20261118 14:58:31.180280 2323 leader_election.cc:304] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d; no voters: 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:31.180573 2569 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 7 FOLLOWER]: Leader election won for term 7
I20261118 14:58:31.180997 2569 raft_consensus.cc:705] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [term 7 LEADER]: Becoming Leader. State: Replica: 0ed65a3719634e519d783540afd6189e, State: Running, Role: LEADER
I20261118 14:58:31.181406 2569 consensus_queue.cc:238] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 112, Committed index: 112, Last appended: 6.112, Last appended by leader: 112, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:31.183034 2576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ed65a3719634e519d783540afd6189e. Latest consensus state: current_term: 7 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:31.183413 2576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:31.185449 2509 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Refusing update from remote peer 0ed65a3719634e519d783540afd6189e: Log matching property violated. Preceding OpId in replica: term: 3 index: 97. Preceding OpId from leader: term: 7 index: 113. (index mismatch)
I20261118 14:58:31.185750 2580 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
W20261118 14:58:31.186141 2324 consensus_peers.cc:597] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e -> Peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Couldn't send request to peer 9b90720bc0b34e4994abf4fc6f9e59e1. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:31.186174 2576 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 113, Last known committed idx: 97, Time since last communication: 0.000s
I20261118 14:58:31.189388 2580 catalog_manager.cc:694] Loaded metadata for table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:31.189962 2580 catalog_manager.cc:694] Loaded metadata for table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:31.190343 2580 catalog_manager.cc:694] Loaded metadata for table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:31.190704 2580 catalog_manager.cc:694] Loaded metadata for table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]
I20261118 14:58:31.191161 2580 catalog_manager.cc:694] Loaded metadata for table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:31.191488 2580 catalog_manager.cc:694] Loaded metadata for table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:31.191946 2580 catalog_manager.cc:694] Loaded metadata for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:31.192284 2580 catalog_manager.cc:694] Loaded metadata for table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]
I20261118 14:58:31.192584 2580 catalog_manager.cc:694] Loaded metadata for table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]
I20261118 14:58:31.192909 2580 catalog_manager.cc:694] Loaded metadata for table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]
I20261118 14:58:31.193396 2580 catalog_manager.cc:694] Loaded metadata for table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]
I20261118 14:58:31.197243 2580 tablet_loader.cc:96] loaded metadata for tablet 2c20495505a1478398a9c447cf18742b (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28])
I20261118 14:58:31.197885 2580 tablet_loader.cc:96] loaded metadata for tablet 42d8a3bab2694c3e83da6a8642d6d911 (table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15])
I20261118 14:58:31.198683 2580 tablet_loader.cc:96] loaded metadata for tablet 58209714937148ad8a3722b5e392bfce (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d])
I20261118 14:58:31.199043 2580 tablet_loader.cc:96] loaded metadata for tablet 6ffda5e2164e4a558231db3043422159 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0])
I20261118 14:58:31.199564 2580 tablet_loader.cc:96] loaded metadata for tablet 77e643a760294dacbfca6305f1811bbd (table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8])
I20261118 14:58:31.201930 2580 tablet_loader.cc:96] loaded metadata for tablet d5ebf8d258a644d9bea96b42357a9345 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b])
I20261118 14:58:31.202283 2580 tablet_loader.cc:96] loaded metadata for tablet da163bd4e25b4254ab3923af361202ce (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7])
I20261118 14:58:31.202842 2580 tablet_loader.cc:96] loaded metadata for tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6])
I20261118 14:58:31.203207 2580 tablet_loader.cc:96] loaded metadata for tablet e47d7d10a5c741ecb30ec701c76b27a3 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f])
I20261118 14:58:31.204043 2580 tablet_loader.cc:96] loaded metadata for tablet f6ce7737d98348659f0c83cb81f7ab45 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8])
I20261118 14:58:31.204421 2580 tablet_loader.cc:96] loaded metadata for tablet f87ee223b5a246c58e9b2eca5740d08c (table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627])
I20261118 14:58:31.204679 2580 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:31.204686 2581 mvcc.cc:204] Tried to move back new op lower bound from 7352376950614847488 to 7352376924339318784. Current Snapshot: MvccSnapshot[applied={T|T < 7352376950614847488 or (T in {7352376950614847488})}]
I20261118 14:58:31.206260 2580 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:31.206346 2580 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:31.206475 2583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:31.206622 2564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ed65a3719634e519d783540afd6189e. Latest consensus state: current_term: 7 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:31.206809 2583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:31.206868 2564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:31.207121 2587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:31.207418 2587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:31.208143 2579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:31.208419 2579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:31.208252 2588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:31.208518 2588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:31.208678 2580 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:31.208747 2587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "0ed65a3719634e519d783540afd6189e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:31.208904 2587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:31.210069 2580 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Loaded TSK: 0
I20261118 14:58:31.210925 2580 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:31.212142 2580 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:31.213364 2580 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 33
I20261118 14:58:31.251626 2337 master_service.cc:965] ReplaceTablet: received request to replace tablet f87ee223b5a246c58e9b2eca5740d08c from {username='slave'} at 127.0.0.1:37730
I20261118 14:58:31.251991 2338 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37730:
table { table_name: "default.Table_4d2c8b2bcb994782a8bbf4b38ca2f7af" } new_table_name: "default.Table_b56407e6850a466082c3b8d7c0dae8d0" modify_external_catalogs: true
I20261118 14:58:31.253068 2333 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37730:
name: "default.Table_6c65bee0620c494591ca0d8d3226c6a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:31.254629 2331 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37730:
table { table_name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9" } modify_external_catalogs: true
I20261118 14:58:31.254509 2336 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37730:
name: "default.Table_d0b14f4adfe3441c88447b2fc1e2d506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:31.254911 2331 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37730:
table { table_name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9" } modify_external_catalogs: true
I20261118 14:58:31.254973 2335 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37730:
name: "default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:31.255182 2330 master_service.cc:965] ReplaceTablet: received request to replace tablet a78ecb3a620242f4a16128b0d456c352 from {username='slave'} at 127.0.0.1:37730
I20261118 14:58:31.255529 2329 master_service.cc:965] ReplaceTablet: received request to replace tablet e47d7d10a5c741ecb30ec701c76b27a3 from {username='slave'} at 127.0.0.1:37730
I20261118 14:58:31.255636 2334 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37730:
name: "default.Table_36b9017c9ae24ecdb8ae81bf85341052"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:31.256035 2330 master_service.cc:965] ReplaceTablet: received request to replace tablet 1b5222f6dc7f4757adbf34c3f6a2729e from {username='slave'} at 127.0.0.1:37730
I20261118 14:58:31.256513 2332 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37730:
name: "default.Table_5c2ab051f54546cab77c75d60b2a8bd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:31.257187 2330 master_service.cc:965] ReplaceTablet: received request to replace tablet 23b3e64e87dd4302a68a8cff81a03da1 from {username='slave'} at 127.0.0.1:37730
I20261118 14:58:31.257858 2330 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37730:
name: "default.Table_4dba69207cca45529e828162200c13f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
F20261118 14:58:31.259351 2392 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 36, previous 36; dumping first 4 out of 5 received events: NotificationEvent(eventId=34, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_4789eba8c66842f6ae326364062589fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_4789eba8c66842f6ae326364062589fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4789eba8c66842f6ae326364062589fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4789eba8c66842f6ae326364062589fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4789eba8c66842f6ae326364062589fb\",\"kudu.table_id\":\"0ce3ba58cd7440a9b27427e7713af331\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=35, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_7171b2f0b6ea470393bf9aa28246cf65, message={"server":"","servicePrincipal":"","db":"default","table":"table_7171b2f0b6ea470393bf9aa28246cf65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_7171b2f0b6ea470393bf9aa28246cf65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_7171b2f0b6ea470393bf9aa28246cf65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_7171b2f0b6ea470393bf9aa28246cf65\",\"kudu.table_id\":\"cdc2cda186804cd39b0d320f08e91df8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_4d2c8b2bcb994782a8bbf4b38ca2f7af, message={"server":"","servicePrincipal":"","db":"default","table":"table_4d2c8b2bcb994782a8bbf4b38ca2f7af","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af\",\"kudu.table_id\":\"d65d9ba97cda44e3bdff3add1fec1ec8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_36b9017c9ae24ecdb8ae81bf85341052, message={"server":"","servicePrincipal":"","db":"default","table":"table_36b9017c9ae24ecdb8ae81bf85341052","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_36b9017c9ae24ecdb8ae81bf85341052\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_36b9017c9ae24ecdb8ae81bf85341052\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_36b9017c9ae24ecdb8ae81bf85341052\",\"kudu.table_id\":\"378bd168f216477992c1b7bb33791778\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f5f170f6d5d google::LogMessage::Fail() at ??:0
I20261118 14:58:31.263211 2329 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Sending DeleteTablet for 3 replicas of tablet e47d7d10a5c741ecb30ec701c76b27a3
@ 0x7f5f170fab23 google::LogMessage::SendToLog() at ??:0
I20261118 14:58:31.264928 2337 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 0ed65a3719634e519d783540afd6189e: Sending DeleteTablet for 3 replicas of tablet f87ee223b5a246c58e9b2eca5740d08c
I20261118 14:58:31.265074 2329 catalog_manager.cc:6936] ReplaceTablet: tablet e47d7d10a5c741ecb30ec701c76b27a3 deleted and replaced by tablet 6a249f4f523e414caa8a762c839e9129
I20261118 14:58:31.265331 2337 catalog_manager.cc:6936] ReplaceTablet: tablet f87ee223b5a246c58e9b2eca5740d08c deleted and replaced by tablet ad6f4e435e914055a4ee7d5afa5e5f6a
@ 0x7f5f170f675c google::LogMessage::Flush() at ??:0
I20261118 14:58:31.265420 2329 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37730:
name: "default.Table_053874e1edfe4b8eac52b4ba5f1b1870"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:31.265623 2337 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37730:
name: "default.Table_d24daba84f8c4c94824da319564f0453"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f5f170f69a1 google::LogMessage::~LogMessage() at ??:0
@ 0x7f5f2c5769a4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
@ 0x7f5f2c575251 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 14:58:31.287302 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet e47d7d10a5c741ecb30ec701c76b27a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a249f4f523e414caa8a762c839e9129) from {username='slave'} at 127.0.0.1:45254
I20261118 14:58:31.287351 1056 tablet_service.cc:1547] Processing DeleteTablet for tablet f87ee223b5a246c58e9b2eca5740d08c with delete_type TABLET_DATA_DELETED (replaced by tablet ad6f4e435e914055a4ee7d5afa5e5f6a) from {username='slave'} at 127.0.0.1:45254
I20261118 14:58:31.287920 2596 tablet_replica.cc:333] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:31.288106 2596 raft_consensus.cc:2261] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:31.288321 2596 raft_consensus.cc:2290] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:31.288707 915 tablet_service.cc:1547] Processing DeleteTablet for tablet e47d7d10a5c741ecb30ec701c76b27a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a249f4f523e414caa8a762c839e9129) from {username='slave'} at 127.0.0.1:37318
I20261118 14:58:31.288883 2596 ts_tablet_manager.cc:1962] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:31.289031 919 tablet_service.cc:1547] Processing DeleteTablet for tablet f87ee223b5a246c58e9b2eca5740d08c with delete_type TABLET_DATA_DELETED (replaced by tablet ad6f4e435e914055a4ee7d5afa5e5f6a) from {username='slave'} at 127.0.0.1:37318
I20261118 14:58:31.291414 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet e47d7d10a5c741ecb30ec701c76b27a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 6a249f4f523e414caa8a762c839e9129) from {username='slave'} at 127.0.0.1:38748
I20261118 14:58:31.291531 1190 tablet_service.cc:1547] Processing DeleteTablet for tablet f87ee223b5a246c58e9b2eca5740d08c with delete_type TABLET_DATA_DELETED (replaced by tablet ad6f4e435e914055a4ee7d5afa5e5f6a) from {username='slave'} at 127.0.0.1:38748
I20261118 14:58:31.299467 1190 tablet_service.cc:1500] Processing CreateTablet for tablet f6ce7737d98348659f0c83cb81f7ab45 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.299662 1053 tablet_service.cc:1500] Processing CreateTablet for tablet 6a249f4f523e414caa8a762c839e9129 (DEFAULT_TABLE table=default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.299680 919 tablet_service.cc:1500] Processing CreateTablet for tablet f6ce7737d98348659f0c83cb81f7ab45 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.299872 1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ce7737d98348659f0c83cb81f7ab45. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.299990 1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a249f4f523e414caa8a762c839e9129. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.300110 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ce7737d98348659f0c83cb81f7ab45. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.301916 914 tablet_service.cc:1500] Processing CreateTablet for tablet 58209714937148ad8a3722b5e392bfce (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.302376 914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58209714937148ad8a3722b5e392bfce. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.302615 1189 tablet_service.cc:1500] Processing CreateTablet for tablet 6a249f4f523e414caa8a762c839e9129 (DEFAULT_TABLE table=default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.302619 917 tablet_service.cc:1500] Processing CreateTablet for tablet 6a249f4f523e414caa8a762c839e9129 (DEFAULT_TABLE table=default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.302939 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a249f4f523e414caa8a762c839e9129. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.303036 917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a249f4f523e414caa8a762c839e9129. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.306073 1056 tablet_service.cc:1500] Processing CreateTablet for tablet f6ce7737d98348659f0c83cb81f7ab45 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.306419 1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ce7737d98348659f0c83cb81f7ab45. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.306638 1192 tablet_service.cc:1500] Processing CreateTablet for tablet 2c20495505a1478398a9c447cf18742b (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.306643 918 tablet_service.cc:1500] Processing CreateTablet for tablet 2c20495505a1478398a9c447cf18742b (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.306955 1192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c20495505a1478398a9c447cf18742b. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.306967 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c20495505a1478398a9c447cf18742b. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f5f2c57cecc kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 14:58:31.310617 1055 tablet_service.cc:1500] Processing CreateTablet for tablet 2c20495505a1478398a9c447cf18742b (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.310647 916 tablet_service.cc:1500] Processing CreateTablet for tablet ad6f4e435e914055a4ee7d5afa5e5f6a (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.310953 916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6f4e435e914055a4ee7d5afa5e5f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.310935 1057 tablet_service.cc:1500] Processing CreateTablet for tablet ad6f4e435e914055a4ee7d5afa5e5f6a (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.311424 1054 tablet_service.cc:1500] Processing CreateTablet for tablet 58209714937148ad8a3722b5e392bfce (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.311497 1187 tablet_service.cc:1500] Processing CreateTablet for tablet ad6f4e435e914055a4ee7d5afa5e5f6a (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.311851 1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6f4e435e914055a4ee7d5afa5e5f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.311944 1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c20495505a1478398a9c447cf18742b. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.314487 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6f4e435e914055a4ee7d5afa5e5f6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.314625 1191 tablet_service.cc:1500] Processing CreateTablet for tablet 58209714937148ad8a3722b5e392bfce (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:31.314918 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58209714937148ad8a3722b5e392bfce. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:31.318480 1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58209714937148ad8a3722b5e392bfce. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f5f2c57cd49 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x372c7d std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 14:58:31.335304 2598 tablet_replica.cc:333] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:31.335511 2598 raft_consensus.cc:2261] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:31.335740 2598 raft_consensus.cc:2290] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:31.336345 2598 ts_tablet_manager.cc:1962] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f5f1853fdc9 kudu::Thread::SuperviseThread() at ??:0
I20261118 14:58:31.344151 2599 tablet_replica.cc:333] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:31.344354 2599 raft_consensus.cc:2261] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:31.344722 2599 raft_consensus.cc:2290] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:31.345299 2599 ts_tablet_manager.cc:1962] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:31.346494 2596 ts_tablet_manager.cc:1975] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:31.346604 2596 log.cc:1199] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/f87ee223b5a246c58e9b2eca5740d08c
I20261118 14:58:31.347013 2596 ts_tablet_manager.cc:1996] T f87ee223b5a246c58e9b2eca5740d08c P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:31.348035 2325 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet f87ee223b5a246c58e9b2eca5740d08c (table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]) successfully deleted
I20261118 14:58:31.348351 2602 tablet_bootstrap.cc:492] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:31.349232 2604 tablet_bootstrap.cc:492] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:31.349927 2601 tablet_bootstrap.cc:492] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:31.351223 2601 tablet_bootstrap.cc:654] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.352336 2599 ts_tablet_manager.cc:1975] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:31.352447 2599 log.cc:1199] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/e47d7d10a5c741ecb30ec701c76b27a3
I20261118 14:58:31.352865 2599 ts_tablet_manager.cc:1996] T e47d7d10a5c741ecb30ec701c76b27a3 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
@ 0x7f5f1bc1a6db start_thread at ??:0
I20261118 14:58:31.353791 2599 tablet_replica.cc:333] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:31.353920 2324 catalog_manager.cc:5065] TS c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343): tablet e47d7d10a5c741ecb30ec701c76b27a3 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]) successfully deleted
I20261118 14:58:31.353978 2599 raft_consensus.cc:2261] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Raft consensus shutting down.
@ 0x7f5f1468a61f clone at ??:0
I20261118 14:58:31.354293 2599 raft_consensus.cc:2290] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:31.354422 2604 tablet_bootstrap.cc:654] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.355057 2599 ts_tablet_manager.cc:1962] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:31.356485 2602 tablet_bootstrap.cc:654] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.358007 2596 tablet_replica.cc:333] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:31.358208 2596 raft_consensus.cc:2261] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:31.358393 2596 raft_consensus.cc:2290] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:31.358541 2601 tablet_bootstrap.cc:492] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:31.358652 2601 ts_tablet_manager.cc:1449] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20261118 14:58:31.358902 2596 ts_tablet_manager.cc:1962] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:31.359364 2601 raft_consensus.cc:360] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.359565 2601 raft_consensus.cc:386] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.359639 2601 raft_consensus.cc:748] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.359845 2598 ts_tablet_manager.cc:1975] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:31.359947 2598 log.cc:1199] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/e47d7d10a5c741ecb30ec701c76b27a3
I20261118 14:58:31.359921 2601 consensus_queue.cc:261] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.360801 2599 ts_tablet_manager.cc:1975] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:31.360910 2599 log.cc:1199] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/f87ee223b5a246c58e9b2eca5740d08c
I20261118 14:58:31.361299 2599 ts_tablet_manager.cc:1996] T f87ee223b5a246c58e9b2eca5740d08c P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:31.362303 2601 ts_tablet_manager.cc:1480] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20261118 14:58:31.362617 2601 tablet_bootstrap.cc:492] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:31.364041 2602 tablet_bootstrap.cc:492] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:31.364171 2602 ts_tablet_manager.cc:1449] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20261118 14:58:31.364256 2601 tablet_bootstrap.cc:654] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.364614 2598 ts_tablet_manager.cc:1996] T e47d7d10a5c741ecb30ec701c76b27a3 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:31.364941 2602 raft_consensus.cc:360] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.365116 2602 raft_consensus.cc:386] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.365211 2602 raft_consensus.cc:748] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.365424 2602 consensus_queue.cc:261] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.365613 2598 tablet_replica.cc:333] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:31.365937 2602 ts_tablet_manager.cc:1480] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261118 14:58:31.366073 2604 tablet_bootstrap.cc:492] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:31.366191 2604 ts_tablet_manager.cc:1449] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20261118 14:58:31.366238 2602 tablet_bootstrap.cc:492] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:31.366873 2604 raft_consensus.cc:360] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.367038 2604 raft_consensus.cc:386] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.367112 2604 raft_consensus.cc:748] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.367329 2604 consensus_queue.cc:261] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.367786 2596 ts_tablet_manager.cc:1975] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:31.367839 2602 tablet_bootstrap.cc:654] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.367897 2596 log.cc:1199] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/e47d7d10a5c741ecb30ec701c76b27a3
I20261118 14:58:31.368261 2596 ts_tablet_manager.cc:1996] T e47d7d10a5c741ecb30ec701c76b27a3 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:31.370929 2598 raft_consensus.cc:2261] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:31.371114 2598 raft_consensus.cc:2290] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:31.371399 2604 ts_tablet_manager.cc:1480] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20261118 14:58:31.371639 2598 ts_tablet_manager.cc:1962] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:31.373171 2604 tablet_bootstrap.cc:492] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:31.374779 2604 tablet_bootstrap.cc:654] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.376791 2604 tablet_bootstrap.cc:492] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:31.376927 2604 ts_tablet_manager.cc:1449] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20261118 14:58:31.377660 2604 raft_consensus.cc:360] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.377823 2604 raft_consensus.cc:386] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.377890 2604 raft_consensus.cc:748] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.378083 2598 ts_tablet_manager.cc:1975] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:31.378199 2598 log.cc:1199] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/f87ee223b5a246c58e9b2eca5740d08c
I20261118 14:58:31.378106 2604 consensus_queue.cc:261] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.378242 2601 tablet_bootstrap.cc:492] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:31.378588 2598 ts_tablet_manager.cc:1996] T f87ee223b5a246c58e9b2eca5740d08c P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:31.378612 2604 ts_tablet_manager.cc:1480] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:31.378888 2604 tablet_bootstrap.cc:492] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:31.380595 2604 tablet_bootstrap.cc:654] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.381561 2601 ts_tablet_manager.cc:1449] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20261118 14:58:31.382323 2601 raft_consensus.cc:360] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.382521 2601 raft_consensus.cc:386] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.382588 2601 raft_consensus.cc:748] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.383117 2602 tablet_bootstrap.cc:492] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:31.382995 2601 consensus_queue.cc:261] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.383246 2602 ts_tablet_manager.cc:1449] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.005s
I20261118 14:58:31.383489 2601 ts_tablet_manager.cc:1480] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20261118 14:58:31.383785 2601 tablet_bootstrap.cc:492] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:31.383960 2602 raft_consensus.cc:360] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.384133 2602 raft_consensus.cc:386] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.384215 2602 raft_consensus.cc:748] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.384429 2602 consensus_queue.cc:261] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.384889 2602 ts_tablet_manager.cc:1480] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20261118 14:58:31.385242 2602 tablet_bootstrap.cc:492] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:31.386989 2602 tablet_bootstrap.cc:654] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.388756 2602 tablet_bootstrap.cc:492] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:31.388885 2602 ts_tablet_manager.cc:1449] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20261118 14:58:31.389438 2602 raft_consensus.cc:360] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.389565 2602 raft_consensus.cc:386] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.389645 2602 raft_consensus.cc:748] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.389838 2602 consensus_queue.cc:261] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.390394 2602 ts_tablet_manager.cc:1480] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20261118 14:58:31.390690 2602 tablet_bootstrap.cc:492] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:31.392270 2602 tablet_bootstrap.cc:654] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.392557 2601 tablet_bootstrap.cc:654] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.395176 2602 tablet_bootstrap.cc:492] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:31.395322 2602 ts_tablet_manager.cc:1449] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20261118 14:58:31.395367 2604 tablet_bootstrap.cc:492] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:31.395490 2601 tablet_bootstrap.cc:492] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:31.395500 2604 ts_tablet_manager.cc:1449] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20261118 14:58:31.395613 2601 ts_tablet_manager.cc:1449] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20261118 14:58:31.395967 2602 raft_consensus.cc:360] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.396130 2602 raft_consensus.cc:386] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.396198 2602 raft_consensus.cc:748] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.396235 2604 raft_consensus.cc:360] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.396278 2601 raft_consensus.cc:360] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.396427 2604 raft_consensus.cc:386] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.396481 2601 raft_consensus.cc:386] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.396512 2604 raft_consensus.cc:748] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.396433 2602 consensus_queue.cc:261] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.396557 2601 raft_consensus.cc:748] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.397008 2602 ts_tablet_manager.cc:1480] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20261118 14:58:31.397343 2602 tablet_bootstrap.cc:492] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:31.398535 2601 consensus_queue.cc:261] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.399001 2602 tablet_bootstrap.cc:654] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.396724 2604 consensus_queue.cc:261] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.400830 2602 tablet_bootstrap.cc:492] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:31.400947 2602 ts_tablet_manager.cc:1449] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20261118 14:58:31.400928 2601 ts_tablet_manager.cc:1480] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20261118 14:58:31.401620 2602 raft_consensus.cc:360] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.401772 2602 raft_consensus.cc:386] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.401851 2602 raft_consensus.cc:748] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.402084 2602 consensus_queue.cc:261] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.403290 2601 tablet_bootstrap.cc:492] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:31.403884 2604 ts_tablet_manager.cc:1480] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20261118 14:58:31.404378 2602 ts_tablet_manager.cc:1480] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20261118 14:58:31.406626 2601 tablet_bootstrap.cc:654] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.406874 2604 tablet_bootstrap.cc:492] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:31.408669 2604 tablet_bootstrap.cc:654] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.411159 2604 tablet_bootstrap.cc:492] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:31.411265 2601 tablet_bootstrap.cc:492] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:31.411298 2604 ts_tablet_manager.cc:1449] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20261118 14:58:31.411397 2601 ts_tablet_manager.cc:1449] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20261118 14:58:31.411975 2604 raft_consensus.cc:360] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.411995 2601 raft_consensus.cc:360] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.412151 2604 raft_consensus.cc:386] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.412151 2601 raft_consensus.cc:386] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.412213 2604 raft_consensus.cc:748] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.412227 2601 raft_consensus.cc:748] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.412469 2604 consensus_queue.cc:261] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.412453 2601 consensus_queue.cc:261] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.413535 2601 ts_tablet_manager.cc:1480] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:31.413808 2601 tablet_bootstrap.cc:492] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:31.414764 2604 ts_tablet_manager.cc:1480] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:31.415037 2604 tablet_bootstrap.cc:492] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:31.415342 2601 tablet_bootstrap.cc:654] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.416718 2604 tablet_bootstrap.cc:654] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:31.423547 2601 tablet_bootstrap.cc:492] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:31.423666 2601 ts_tablet_manager.cc:1449] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20261118 14:58:31.423970 2604 tablet_bootstrap.cc:492] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:31.424082 2604 ts_tablet_manager.cc:1449] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20261118 14:58:31.424345 2601 raft_consensus.cc:360] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.424495 2601 raft_consensus.cc:386] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.424573 2601 raft_consensus.cc:748] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.424751 2604 raft_consensus.cc:360] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.424804 2601 consensus_queue.cc:261] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.424894 2604 raft_consensus.cc:386] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:31.424966 2604 raft_consensus.cc:748] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:31.425393 2601 ts_tablet_manager.cc:1480] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:31.426631 2604 consensus_queue.cc:261] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.427932 2604 ts_tablet_manager.cc:1480] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20261118 14:58:31.467710 2603 raft_consensus.cc:499] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:31.467878 2603 raft_consensus.cc:521] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.468788 2603 leader_election.cc:290] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:31.469161 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "6a249f4f523e414caa8a762c839e9129" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:31.469269 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "6a249f4f523e414caa8a762c839e9129" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:31.469386 1210 raft_consensus.cc:2490] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:31.469447 1076 raft_consensus.cc:2490] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:31.469822 874 leader_election.cc:304] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:31.470072 2603 raft_consensus.cc:2826] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:31.470180 2603 raft_consensus.cc:499] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:31.470263 2603 raft_consensus.cc:3096] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.472146 2603 raft_consensus.cc:521] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.472680 2603 leader_election.cc:290] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:31.473001 1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "6a249f4f523e414caa8a762c839e9129" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:31.473186 1210 raft_consensus.cc:3096] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.473188 1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "6a249f4f523e414caa8a762c839e9129" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:31.473356 1076 raft_consensus.cc:3096] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.476056 1210 raft_consensus.cc:2490] T 6a249f4f523e414caa8a762c839e9129 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:31.476089 1076 raft_consensus.cc:2490] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:31.476437 874 leader_election.cc:304] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:31.476687 2603 raft_consensus.cc:2826] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:31.476823 2603 raft_consensus.cc:705] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:31.477290 2603 consensus_queue.cc:238] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.483570 2606 consensus_queue.cc:1049] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:31.484761 2606 consensus_queue.cc:1049] T 6a249f4f523e414caa8a762c839e9129 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 14:58:31.558169 1010 connection.cc:573] server connection from 127.0.0.1:45254 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:31.558176 875 connection.cc:573] server connection from 127.0.0.1:37318 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:31.558195 1147 connection.cc:573] server connection from 127.0.0.1:38748 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:31.558609 1010 connection.cc:573] client connection to 127.0.72.190:42447 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:31.559108 1145 connection.cc:573] client connection to 127.0.72.190:42447 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:31.559298 1264 connection.cc:573] client connection to 127.0.72.190:42447 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:31.559926 1264 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.190:42447)
W20261118 14:58:31.560515 1264 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.190:42447)
W20261118 14:58:31.560870 1264 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.190:42447)
W20261118 14:58:31.561043 1264 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.190:42447)
W20261118 14:58:31.561712 2559 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 14:58:31.560084 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.190:42447 (local address 127.0.72.130:41061)
1118 14:58:31.560310 (+ 226us) negotiation.cc:107] Waiting for socket to connect
1118 14:58:31.560327 (+ 17us) client_negotiation.cc:175] Beginning negotiation
1118 14:58:31.560448 (+ 121us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 14:58:31.561677 (+ 1229us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":129}
W20261118 14:58:31.561954 2567 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 14:58:31.559329 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.190:42447 (local address 127.0.72.129:46987)
1118 14:58:31.559606 (+ 277us) negotiation.cc:107] Waiting for socket to connect
1118 14:58:31.559624 (+ 18us) client_negotiation.cc:175] Beginning negotiation
1118 14:58:31.559750 (+ 126us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 14:58:31.561914 (+ 2164us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":161}
I20261118 14:58:31.642738 2630 raft_consensus.cc:499] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:31.642953 2630 raft_consensus.cc:521] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.643757 2630 leader_election.cc:290] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:31.644313 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f6ce7737d98348659f0c83cb81f7ab45" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:31.644434 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f6ce7737d98348659f0c83cb81f7ab45" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:31.644542 1211 raft_consensus.cc:2490] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:31.644690 1075 raft_consensus.cc:2490] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:31.645090 874 leader_election.cc:304] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:31.645547 2630 raft_consensus.cc:2826] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:31.645725 2630 raft_consensus.cc:499] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:31.645871 2630 raft_consensus.cc:3096] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.648422 2630 raft_consensus.cc:521] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.648980 2630 leader_election.cc:290] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:31.649612 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f6ce7737d98348659f0c83cb81f7ab45" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:31.649854 1211 raft_consensus.cc:3096] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.650023 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f6ce7737d98348659f0c83cb81f7ab45" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:31.650238 1075 raft_consensus.cc:3096] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.652446 1075 raft_consensus.cc:2490] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:31.652773 1211 raft_consensus.cc:2490] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:31.653506 875 leader_election.cc:304] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:31.654081 2630 raft_consensus.cc:2826] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:31.654557 2630 raft_consensus.cc:705] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:31.655150 2630 consensus_queue.cc:238] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.661850 2606 consensus_queue.cc:1049] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20261118 14:58:31.662817 2603 consensus_queue.cc:1049] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:31.742460 2641 raft_consensus.cc:499] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:31.743003 2641 raft_consensus.cc:521] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.744670 2641 leader_election.cc:290] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:31.745471 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2c20495505a1478398a9c447cf18742b" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:31.745577 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2c20495505a1478398a9c447cf18742b" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:31.745854 938 raft_consensus.cc:2490] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:31.745865 1211 raft_consensus.cc:2490] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:31.746770 1010 leader_election.cc:304] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:31.747224 2641 raft_consensus.cc:2826] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:31.747524 2641 raft_consensus.cc:499] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:31.747605 2641 raft_consensus.cc:3096] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.751394 2641 raft_consensus.cc:521] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.752297 2641 leader_election.cc:290] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:31.753157 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2c20495505a1478398a9c447cf18742b" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:31.753288 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2c20495505a1478398a9c447cf18742b" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:31.753589 938 raft_consensus.cc:3096] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.753613 1211 raft_consensus.cc:3096] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.756446 1211 raft_consensus.cc:2490] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:31.756616 938 raft_consensus.cc:2490] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:31.757184 1012 leader_election.cc:304] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:31.757578 2641 raft_consensus.cc:2826] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:31.757805 2641 raft_consensus.cc:705] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:31.758535 2641 consensus_queue.cc:238] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.762082 2560 consensus_queue.cc:1049] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:31.762800 2560 consensus_queue.cc:1049] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:31.817274 2647 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:31.817519 2647 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:31.818300 2647 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:31.820829 2483 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:31.821095 2485 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
I20261118 14:58:31.821238 2485 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c9ef96ff074450a879e66787486386d; no voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:31.821539 2647 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 14:58:31.902283 1962 raft_consensus.cc:499] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:31.902525 1962 raft_consensus.cc:521] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.903348 1962 leader_election.cc:290] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:31.903865 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "58209714937148ad8a3722b5e392bfce" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:31.903895 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "58209714937148ad8a3722b5e392bfce" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:31.904068 938 raft_consensus.cc:2490] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:31.904070 1075 raft_consensus.cc:2490] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:31.904489 1148 leader_election.cc:304] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:31.904758 1962 raft_consensus.cc:2826] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:31.904860 1962 raft_consensus.cc:499] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:31.904924 1962 raft_consensus.cc:3096] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.906919 1962 raft_consensus.cc:521] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.907591 1962 leader_election.cc:290] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:31.908036 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "58209714937148ad8a3722b5e392bfce" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:31.908059 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "58209714937148ad8a3722b5e392bfce" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:31.908210 938 raft_consensus.cc:3096] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.908226 1075 raft_consensus.cc:3096] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:31.910293 938 raft_consensus.cc:2490] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:31.910578 1075 raft_consensus.cc:2490] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:31.910799 1145 leader_election.cc:304] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:31.911163 1962 raft_consensus.cc:2826] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:31.911505 1962 raft_consensus.cc:705] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:31.911798 1962 consensus_queue.cc:238] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:31.914597 2651 consensus_queue.cc:1049] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:31.915542 2651 consensus_queue.cc:1049] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:32.031751 2640 raft_consensus.cc:499] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:32.031953 2640 raft_consensus.cc:521] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:32.032632 2640 leader_election.cc:290] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:32.033241 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ad6f4e435e914055a4ee7d5afa5e5f6a" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:32.033281 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ad6f4e435e914055a4ee7d5afa5e5f6a" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:32.033509 938 raft_consensus.cc:2490] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:32.033504 1211 raft_consensus.cc:2490] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:32.033979 1012 leader_election.cc:304] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:32.034293 2640 raft_consensus.cc:2826] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:32.034440 2640 raft_consensus.cc:499] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:32.034543 2640 raft_consensus.cc:3096] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:32.036480 2640 raft_consensus.cc:521] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:32.037038 2640 leader_election.cc:290] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:32.037552 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ad6f4e435e914055a4ee7d5afa5e5f6a" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:32.037613 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ad6f4e435e914055a4ee7d5afa5e5f6a" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:32.037796 1211 raft_consensus.cc:3096] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:32.037791 938 raft_consensus.cc:3096] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:32.040807 938 raft_consensus.cc:2490] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:32.040791 1211 raft_consensus.cc:2490] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:32.041445 1010 leader_election.cc:304] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:32.041821 2640 raft_consensus.cc:2826] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:32.042042 2640 raft_consensus.cc:705] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:32.042320 2640 consensus_queue.cc:238] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:32.045115 2640 consensus_queue.cc:1049] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:32.045357 2560 consensus_queue.cc:1049] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:32.432361 2659 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:32.432551 2659 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:32.433342 2659 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:32.435508 2485 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967): Network error: Client connection negotiation failed: client connection to 127.0.72.188:33967: connect: Connection refused (error 111)
W20261118 14:58:32.435690 2483 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:32.435819 2483 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c9ef96ff074450a879e66787486386d; no voters: 0ed65a3719634e519d783540afd6189e, 9b90720bc0b34e4994abf4fc6f9e59e1
I20261118 14:58:32.436058 2659 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 14:58:33.090034 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:33967
--webserver_interface=127.0.72.188
--webserver_port=42503
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 14:58:33.426987 2679 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:33.427281 2679 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:33.427361 2679 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:33.435618 2679 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:33.435721 2679 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:33.435798 2679 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:33.435858 2679 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:33.446875 2679 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.72.188
--webserver_port=42503
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:33.448523 2679 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:33.450341 2679 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:33.458528 2687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:33.458498 2686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:33.458877 2689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:33.460083 2679 server_base.cc:1061] running on GCE node
I20261118 14:58:33.460801 2679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:33.462209 2679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:33.463438 2679 hybrid_clock.cc:648] HybridClock initialized: now 1795013913463369 us; error 49 us; skew 500 ppm
I20261118 14:58:33.463788 2679 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:33.465891 2679 webserver.cc:577] Webserver started at http://127.0.72.188:42503/ using document root <none> and password file <none>
I20261118 14:58:33.466639 2679 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:33.466747 2679 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:33.472007 2679 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20261118 14:58:33.474915 2695 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:33.476521 2679 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20261118 14:58:33.476929 2679 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:33.477793 2679 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:33.499301 2679 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:33.500137 2679 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:33.500401 2679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:33.519104 2679 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.188:33967
I20261118 14:58:33.519127 2746 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.188:33967 every 8 connection(s)
I20261118 14:58:33.520795 2679 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-2/data/info.pb
I20261118 14:58:33.527585 2747 sys_catalog.cc:263] Verifying existing consensus state
I20261118 14:58:33.527974 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 2679
I20261118 14:58:33.531580 2747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap starting.
I20261118 14:58:33.551385 2747 log.cc:826] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:33.553344 2751 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:33.553504 2751 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:33.554152 2751 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:33.556612 2483 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:33.597958 984 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:33.605218 1257 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:33.655843 2747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=112 overwritten=0 applied=110 ignored=0} inserts{seen=70 ignored=0} mutations{seen=97 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20261118 14:58:33.656670 2747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: Bootstrap complete.
I20261118 14:58:33.664347 2747 raft_consensus.cc:360] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 6 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:33.667768 2747 raft_consensus.cc:748] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b90720bc0b34e4994abf4fc6f9e59e1, State: Initialized, Role: FOLLOWER
I20261118 14:58:33.668551 2747 consensus_queue.cc:261] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 110, Last appended: 6.112, Last appended by leader: 112, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:33.669363 2762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:33.669655 2762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:33.670392 2747 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:33.670449 2722 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 8 candidate_status { last_received { term: 7 index: 116 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:33.670959 2722 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9ef96ff074450a879e66787486386d in term 6.
I20261118 14:58:33.671589 2485 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e
I20261118 14:58:33.671974 2751 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Leader pre-election won for term 8
I20261118 14:58:33.672101 2751 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0ed65a3719634e519d783540afd6189e)
I20261118 14:58:33.672206 2751 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 7 FOLLOWER]: Advancing to term 8
I20261118 14:58:33.674682 2751 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:33.675405 2751 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 election: Requested vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:33.676199 2722 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 8 candidate_status { last_received { term: 7 index: 116 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
I20261118 14:58:33.676467 2722 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 6 FOLLOWER]: Advancing to term 8
W20261118 14:58:33.676791 2483 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:33.680483 2722 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9ef96ff074450a879e66787486386d in term 8.
I20261118 14:58:33.681008 2485 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e
I20261118 14:58:33.681356 2751 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Leader election won for term 8
I20261118 14:58:33.681794 2751 raft_consensus.cc:705] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 LEADER]: Becoming Leader. State: Replica: 7c9ef96ff074450a879e66787486386d, State: Running, Role: LEADER
I20261118 14:58:33.682318 2751 consensus_queue.cc:238] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 116, Committed index: 116, Last appended: 7.116, Last appended by leader: 116, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:33.684194 2773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c9ef96ff074450a879e66787486386d. Latest consensus state: current_term: 8 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:33.684458 2773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:33.695708 2780 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 14:58:33.697737 2722 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Refusing update from remote peer 7c9ef96ff074450a879e66787486386d: Log matching property violated. Preceding OpId in replica: term: 6 index: 112. Preceding OpId from leader: term: 8 index: 117. (index mismatch)
I20261118 14:58:33.698601 2773 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 117, Last known committed idx: 110, Time since last communication: 0.000s
I20261118 14:58:33.709615 2780 catalog_manager.cc:694] Loaded metadata for table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:33.711871 2780 catalog_manager.cc:694] Loaded metadata for table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:33.712486 2780 catalog_manager.cc:694] Loaded metadata for table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:33.712955 2780 catalog_manager.cc:694] Loaded metadata for table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]
I20261118 14:58:33.713618 2780 catalog_manager.cc:694] Loaded metadata for table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:33.714041 2780 catalog_manager.cc:694] Loaded metadata for table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:33.714730 2780 catalog_manager.cc:694] Loaded metadata for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:33.715180 2780 catalog_manager.cc:694] Loaded metadata for table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]
I20261118 14:58:33.715615 2780 catalog_manager.cc:694] Loaded metadata for table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]
I20261118 14:58:33.716040 2780 catalog_manager.cc:694] Loaded metadata for table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]
I20261118 14:58:33.716642 2780 catalog_manager.cc:694] Loaded metadata for table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]
W20261118 14:58:33.721361 2483 consensus_peers.cc:597] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d -> Peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Couldn't send request to peer 0ed65a3719634e519d783540afd6189e. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:33.724514 2780 tablet_loader.cc:96] loaded metadata for tablet 2c20495505a1478398a9c447cf18742b (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28])
I20261118 14:58:33.725200 2780 tablet_loader.cc:96] loaded metadata for tablet 42d8a3bab2694c3e83da6a8642d6d911 (table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15])
I20261118 14:58:33.726387 2780 tablet_loader.cc:96] loaded metadata for tablet 58209714937148ad8a3722b5e392bfce (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d])
I20261118 14:58:33.726838 2780 tablet_loader.cc:96] loaded metadata for tablet 6a249f4f523e414caa8a762c839e9129 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f])
I20261118 14:58:33.727286 2780 tablet_loader.cc:96] loaded metadata for tablet 6ffda5e2164e4a558231db3043422159 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0])
I20261118 14:58:33.727931 2780 tablet_loader.cc:96] loaded metadata for tablet 77e643a760294dacbfca6305f1811bbd (table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8])
I20261118 14:58:33.730610 2780 tablet_loader.cc:96] loaded metadata for tablet ad6f4e435e914055a4ee7d5afa5e5f6a (table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627])
I20261118 14:58:33.732204 2780 tablet_loader.cc:96] loaded metadata for tablet d5ebf8d258a644d9bea96b42357a9345 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b])
I20261118 14:58:33.732708 2780 tablet_loader.cc:96] loaded metadata for tablet da163bd4e25b4254ab3923af361202ce (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7])
I20261118 14:58:33.734108 2780 tablet_loader.cc:96] loaded metadata for tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6])
I20261118 14:58:33.735268 2780 tablet_loader.cc:96] loaded metadata for tablet f6ce7737d98348659f0c83cb81f7ab45 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8])
I20261118 14:58:33.736011 2780 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:33.738113 2780 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:33.738267 2780 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:33.740675 2780 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:33.742627 2780 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Loaded TSK: 0
I20261118 14:58:33.743952 2780 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:33.746565 2780 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:33.748533 2780 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 33
I20261118 14:58:33.756474 1122 heartbeater.cc:345] Connected to a master server at 127.0.72.188:33967
I20261118 14:58:33.756639 1122 heartbeater.cc:500] Master 127.0.72.188:33967 was elected leader, sending a full tablet report...
I20261118 14:58:33.753859 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:58:33.760172 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:33.763290 2762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c9ef96ff074450a879e66787486386d. Latest consensus state: current_term: 8 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:33.763530 2762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:33.765204 2762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:33.765376 2762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:33.765867 2786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:33.766208 2786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:33.766845 2751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 8 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:33.767059 2751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:33.767176 2710 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" instance_seqno: 1795013888201158) as {username='slave'} at 127.0.72.130:40877; Asking this server to re-register.
I20261118 14:58:33.767969 1122 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:33.767999 2761 mvcc.cc:204] Tried to move back new op lower bound from 7352376980631224320 to 7352376980202336256. Current Snapshot: MvccSnapshot[applied={T|T < 7352376980631224320}]
I20261118 14:58:33.770022 2712 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:33.772415 2777 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:33.772511 2777 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: loading cluster ID for follower catalog manager: success
I20261118 14:58:33.776993 2777 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: acquiring CA information for follower catalog manager: success
I20261118 14:58:33.779390 2777 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:33.812788 1124 heartbeater.cc:500] Master 127.0.72.189:46707 was elected leader, sending a full tablet report...
I20261118 14:58:33.818635 2498 catalog_manager.cc:5840] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c4fe5e44448d98db1eed6329d150d (127.0.72.130). New cstate: current_term: 1 leader_uuid: "245c4fe5e44448d98db1eed6329d150d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: HEALTHY } } }
I20261118 14:58:33.819651 2498 catalog_manager.cc:5840] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129). New cstate: current_term: 1 leader_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } }
I20261118 14:58:33.820106 2498 catalog_manager.cc:5840] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 245c4fe5e44448d98db1eed6329d150d (127.0.72.130). New cstate: current_term: 1 leader_uuid: "245c4fe5e44448d98db1eed6329d150d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } health_report { overall_health: HEALTHY } } }
I20261118 14:58:33.820487 2498 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet 2c20495505a1478398a9c447cf18742b (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]). Expected version 1 got 0
I20261118 14:58:33.820940 2499 master_service.cc:965] ReplaceTablet: received request to replace tablet cafee7e386434b3c85542d7b6c5024ed from {username='slave'} at 127.0.0.1:38192
I20261118 14:58:33.821394 2495 master_service.cc:965] ReplaceTablet: received request to replace tablet ad6f4e435e914055a4ee7d5afa5e5f6a from {username='slave'} at 127.0.0.1:38192
I20261118 14:58:33.821272 2498 catalog_manager.cc:5840] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "c01f1b88968f4905ba5a401a81ade9f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } }
I20261118 14:58:33.821597 2498 catalog_manager.cc:5940] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257) does not have the latest schema for tablet 58209714937148ad8a3722b5e392bfce (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]). Expected version 2 got 0
I20261118 14:58:33.821861 2498 catalog_manager.cc:5840] T 6a249f4f523e414caa8a762c839e9129 P 245c4fe5e44448d98db1eed6329d150d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129). New cstate: current_term: 1 leader_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } }
I20261118 14:58:33.822296 2490 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38192:
table { table_name: "default.Table_a0ea13e08f4e45bc84d5936c615c8ff7" } new_table_name: "default.Table_8afe8b6b4fe4433882ded7831ce722e9" modify_external_catalogs: true
I20261118 14:58:33.823241 2491 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38192:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:33.823475 2491 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38192:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:33.823907 2492 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38192:
table { table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" } modify_external_catalogs: true
I20261118 14:58:33.824138 2492 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38192:
table { table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" } modify_external_catalogs: true
I20261118 14:58:33.824561 2493 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38192:
table { table_name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1" } modify_external_catalogs: true
I20261118 14:58:33.824772 2493 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38192:
table { table_name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1" } modify_external_catalogs: true
I20261118 14:58:33.825081 2494 master_service.cc:965] ReplaceTablet: received request to replace tablet 23b3e64e87dd4302a68a8cff81a03da1 from {username='slave'} at 127.0.0.1:38192
I20261118 14:58:33.825377 2496 master_service.cc:965] ReplaceTablet: received request to replace tablet 1b5222f6dc7f4757adbf34c3f6a2729e from {username='slave'} at 127.0.0.1:38192
I20261118 14:58:33.825662 2497 master_service.cc:965] ReplaceTablet: received request to replace tablet f6ce7737d98348659f0c83cb81f7ab45 from {username='slave'} at 127.0.0.1:38192
I20261118 14:58:33.826390 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet cafee7e386434b3c85542d7b6c5024ed with delete_type TABLET_DATA_DELETED (replaced by tablet 77e643a760294dacbfca6305f1811bbd) from {username='slave'} at 127.0.0.1:45256
F20261118 14:58:33.827697 2552 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 36, previous 36; dumping first 4 out of 5 received events: NotificationEvent(eventId=34, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_4789eba8c66842f6ae326364062589fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_4789eba8c66842f6ae326364062589fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4789eba8c66842f6ae326364062589fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4789eba8c66842f6ae326364062589fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4789eba8c66842f6ae326364062589fb\",\"kudu.table_id\":\"0ce3ba58cd7440a9b27427e7713af331\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=35, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_7171b2f0b6ea470393bf9aa28246cf65, message={"server":"","servicePrincipal":"","db":"default","table":"table_7171b2f0b6ea470393bf9aa28246cf65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_7171b2f0b6ea470393bf9aa28246cf65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_7171b2f0b6ea470393bf9aa28246cf65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_7171b2f0b6ea470393bf9aa28246cf65\",\"kudu.table_id\":\"cdc2cda186804cd39b0d320f08e91df8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_4d2c8b2bcb994782a8bbf4b38ca2f7af, message={"server":"","servicePrincipal":"","db":"default","table":"table_4d2c8b2bcb994782a8bbf4b38ca2f7af","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af\",\"kudu.table_id\":\"d65d9ba97cda44e3bdff3add1fec1ec8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_36b9017c9ae24ecdb8ae81bf85341052, message={"server":"","servicePrincipal":"","db":"default","table":"table_36b9017c9ae24ecdb8ae81bf85341052","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_36b9017c9ae24ecdb8ae81bf85341052\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_36b9017c9ae24ecdb8ae81bf85341052\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_36b9017c9ae24ecdb8ae81bf85341052\",\"kudu.table_id\":\"378bd168f216477992c1b7bb33791778\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
@ 0x7f238c9f4d5d google::LogMessage::Fail() at ??:0
@ 0x7f238c9f8b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f238c9f475c google::LogMessage::Flush() at ??:0
@ 0x7f238c9f49a1 google::LogMessage::~LogMessage() at ??:0
I20261118 14:58:33.844455 2794 tablet_replica.cc:333] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:33.844678 2794 raft_consensus.cc:2261] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:33.845096 2794 raft_consensus.cc:2290] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f23a1e749a4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 14:58:33.845715 2794 ts_tablet_manager.cc:1962] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:33.847363 2496 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38192:
name: "default.Table_0cbce4af337d4a5baeb313a32662d446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:33.848362 2494 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38192:
table { table_name: "default.Table_7171b2f0b6ea470393bf9aa28246cf65" } new_table_name: "default.Table_de4cb9add02543cb98fda40cb17f0587" modify_external_catalogs: true
@ 0x7f23a1e73251 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 14:58:33.851905 2794 ts_tablet_manager.cc:1975] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:33.852010 2794 log.cc:1199] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/cafee7e386434b3c85542d7b6c5024ed
I20261118 14:58:33.852403 2794 ts_tablet_manager.cc:1996] T cafee7e386434b3c85542d7b6c5024ed P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:33.852996 2499 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38192:
name: "default.Table_6c65bee0620c494591ca0d8d3226c6a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:33.853735 2486 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet cafee7e386434b3c85542d7b6c5024ed (table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]) successfully deleted
I20261118 14:58:33.854609 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet 1b5222f6dc7f4757adbf34c3f6a2729e with delete_type TABLET_DATA_DELETED (replaced by tablet 2c20495505a1478398a9c447cf18742b) from {username='slave'} at 127.0.0.1:45256
I20261118 14:58:33.854684 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 23b3e64e87dd4302a68a8cff81a03da1 with delete_type TABLET_DATA_DELETED (replaced by tablet 58209714937148ad8a3722b5e392bfce) from {username='slave'} at 127.0.0.1:45256
I20261118 14:58:33.854945 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 309f067161fe469dbb29d14c16139058 with delete_type TABLET_DATA_DELETED (replaced by tablet 42d8a3bab2694c3e83da6a8642d6d911) from {username='slave'} at 127.0.0.1:45256
I20261118 14:58:33.855185 2794 tablet_replica.cc:333] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:33.855378 2794 raft_consensus.cc:2261] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:33.855543 2794 raft_consensus.cc:2290] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:33.856174 2794 ts_tablet_manager.cc:1962] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f23a1e7aecc kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 14:58:33.858390 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet cafee7e386434b3c85542d7b6c5024ed with delete_type TABLET_DATA_DELETED (replaced by tablet 77e643a760294dacbfca6305f1811bbd) from {username='slave'} at 127.0.0.1:45256
@ 0x7f23a1e7ad49 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x372c7d std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 14:58:33.865557 2495 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet ad6f4e435e914055a4ee7d5afa5e5f6a
@ 0x7f238de3ddc9 kudu::Thread::SuperviseThread() at ??:0
I20261118 14:58:33.868335 2495 catalog_manager.cc:6936] ReplaceTablet: tablet ad6f4e435e914055a4ee7d5afa5e5f6a deleted and replaced by tablet 9cc4915e3e0344f3b262ba85f16d9be4
I20261118 14:58:33.871408 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet ad6f4e435e914055a4ee7d5afa5e5f6a with delete_type TABLET_DATA_DELETED (replaced by tablet 9cc4915e3e0344f3b262ba85f16d9be4) from {username='slave'} at 127.0.0.1:45256
I20261118 14:58:33.871626 1055 tablet_service.cc:1547] Processing DeleteTablet for tablet a78ecb3a620242f4a16128b0d456c352 with delete_type TABLET_DATA_DELETED (replaced by tablet f6ce7737d98348659f0c83cb81f7ab45) from {username='slave'} at 127.0.0.1:45256
I20261118 14:58:33.872927 2497 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet f6ce7737d98348659f0c83cb81f7ab45
I20261118 14:58:33.873432 2497 catalog_manager.cc:6936] ReplaceTablet: tablet f6ce7737d98348659f0c83cb81f7ab45 deleted and replaced by tablet a8655d55cde2410184cf0c74683f8717
I20261118 14:58:33.874046 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet f6ce7737d98348659f0c83cb81f7ab45 with delete_type TABLET_DATA_DELETED (replaced by tablet a8655d55cde2410184cf0c74683f8717) from {username='slave'} at 127.0.0.1:45256
I20261118 14:58:33.878999 2495 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38192:
name: "default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:33.879526 2497 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38192:
name: "default.Table_d0b14f4adfe3441c88447b2fc1e2d506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:33.880174 2794 ts_tablet_manager.cc:1975] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:33.880297 2794 log.cc:1199] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/1b5222f6dc7f4757adbf34c3f6a2729e
I20261118 14:58:33.880729 2794 ts_tablet_manager.cc:1996] T 1b5222f6dc7f4757adbf34c3f6a2729e P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:33.881767 2486 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 1b5222f6dc7f4757adbf34c3f6a2729e (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]) successfully deleted
I20261118 14:58:33.882082 2794 tablet_replica.cc:333] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:33.882277 2794 raft_consensus.cc:2261] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:33.882476 2794 raft_consensus.cc:2290] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:33.883045 2794 ts_tablet_manager.cc:1962] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f23915186db start_thread at ??:0
I20261118 14:58:33.889348 2794 ts_tablet_manager.cc:1975] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:33.889452 2794 log.cc:1199] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/23b3e64e87dd4302a68a8cff81a03da1
I20261118 14:58:33.889850 2794 ts_tablet_manager.cc:1996] T 23b3e64e87dd4302a68a8cff81a03da1 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:33.890863 2486 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 23b3e64e87dd4302a68a8cff81a03da1 (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:33.891196 2794 tablet_replica.cc:333] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:33.891383 2794 raft_consensus.cc:2261] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:33.891532 2794 raft_consensus.cc:2290] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:33.892005 2794 ts_tablet_manager.cc:1962] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:33.893170 2498 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38192:
name: "default.Table_36b9017c9ae24ecdb8ae81bf85341052"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:33.895866 1187 tablet_service.cc:1547] Processing DeleteTablet for tablet ad6f4e435e914055a4ee7d5afa5e5f6a with delete_type TABLET_DATA_DELETED (replaced by tablet 9cc4915e3e0344f3b262ba85f16d9be4) from {username='slave'} at 127.0.0.1:38756
I20261118 14:58:33.896210 1189 tablet_service.cc:1547] Processing DeleteTablet for tablet f6ce7737d98348659f0c83cb81f7ab45 with delete_type TABLET_DATA_DELETED (replaced by tablet a8655d55cde2410184cf0c74683f8717) from {username='slave'} at 127.0.0.1:38756
I20261118 14:58:33.898912 1189 tablet_service.cc:1500] Processing CreateTablet for tablet a8655d55cde2410184cf0c74683f8717 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:33.899338 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8655d55cde2410184cf0c74683f8717. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:33.900717 1057 tablet_service.cc:1500] Processing CreateTablet for tablet a8655d55cde2410184cf0c74683f8717 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:33.901064 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8655d55cde2410184cf0c74683f8717. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:33.908339 1191 tablet_service.cc:1500] Processing CreateTablet for tablet 9cc4915e3e0344f3b262ba85f16d9be4 (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:33.908717 1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc4915e3e0344f3b262ba85f16d9be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:33.909436 1055 tablet_service.cc:1500] Processing CreateTablet for tablet 9cc4915e3e0344f3b262ba85f16d9be4 (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:33.909801 1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc4915e3e0344f3b262ba85f16d9be4. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f2389f8861f clone at ??:0
I20261118 14:58:33.929965 2804 tablet_replica.cc:333] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:33.930192 2804 raft_consensus.cc:2261] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:33.930434 2804 raft_consensus.cc:2290] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:33.931286 915 tablet_service.cc:1547] Processing DeleteTablet for tablet f6ce7737d98348659f0c83cb81f7ab45 with delete_type TABLET_DATA_DELETED (replaced by tablet a8655d55cde2410184cf0c74683f8717) from {username='slave'} at 127.0.0.1:37328
I20261118 14:58:33.931532 914 tablet_service.cc:1547] Processing DeleteTablet for tablet ad6f4e435e914055a4ee7d5afa5e5f6a with delete_type TABLET_DATA_DELETED (replaced by tablet 9cc4915e3e0344f3b262ba85f16d9be4) from {username='slave'} at 127.0.0.1:37328
I20261118 14:58:33.931929 919 tablet_service.cc:1500] Processing CreateTablet for tablet a8655d55cde2410184cf0c74683f8717 (DEFAULT_TABLE table=default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:33.932153 2809 tablet_bootstrap.cc:492] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:33.932356 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8655d55cde2410184cf0c74683f8717. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:33.935029 2808 tablet_bootstrap.cc:492] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:33.937044 2794 ts_tablet_manager.cc:1975] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:33.937044 918 tablet_service.cc:1500] Processing CreateTablet for tablet 9cc4915e3e0344f3b262ba85f16d9be4 (DEFAULT_TABLE table=default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:33.937192 2794 log.cc:1199] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/309f067161fe469dbb29d14c16139058
I20261118 14:58:33.937464 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cc4915e3e0344f3b262ba85f16d9be4. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:33.937615 2794 ts_tablet_manager.cc:1996] T 309f067161fe469dbb29d14c16139058 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:33.938766 2814 tablet.cc:1759] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:33.939951 2804 ts_tablet_manager.cc:1962] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:33.943112 2812 tablet_replica.cc:333] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:33.943334 2812 raft_consensus.cc:2261] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:33.947742 2809 tablet_bootstrap.cc:654] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:33.949486 2812 raft_consensus.cc:2290] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:33.950080 2812 ts_tablet_manager.cc:1962] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:33.954833 2815 tablet.cc:1759] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:33.959278 2818 tablet_bootstrap.cc:492] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:33.961242 2818 tablet_bootstrap.cc:654] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:33.961534 2809 tablet_bootstrap.cc:492] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:33.961685 2809 ts_tablet_manager.cc:1449] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20261118 14:58:33.962493 2809 raft_consensus.cc:360] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:33.966122 2808 tablet_bootstrap.cc:654] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:33.984659 2808 tablet_bootstrap.cc:492] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:33.984658 2818 tablet_bootstrap.cc:492] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:33.984810 2818 ts_tablet_manager.cc:1449] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20261118 14:58:33.984810 2808 ts_tablet_manager.cc:1449] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.009s
I20261118 14:58:33.985558 2808 raft_consensus.cc:360] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:33.985558 2818 raft_consensus.cc:360] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:33.985739 2808 raft_consensus.cc:386] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:33.985749 2818 raft_consensus.cc:386] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:33.985822 2808 raft_consensus.cc:748] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:33.985826 2818 raft_consensus.cc:748] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:33.986042 2808 consensus_queue.cc:261] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:33.986083 2818 consensus_queue.cc:261] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:33.987119 2818 ts_tablet_manager.cc:1480] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:33.989338 2712 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" instance_seqno: 1795013888668865) as {username='slave'} at 127.0.72.131:55263; Asking this server to re-register.
I20261118 14:58:33.990957 1257 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:33.998960 2829 tablet.cc:1759] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20261118 14:58:33.999308 2712 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:34.002136 2804 ts_tablet_manager.cc:1975] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:34.002250 2804 log.cc:1199] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/ad6f4e435e914055a4ee7d5afa5e5f6a
I20261118 14:58:34.002776 2794 tablet_replica.cc:333] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:34.002970 2794 raft_consensus.cc:2261] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:34.003288 2794 raft_consensus.cc:2290] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:34.003877 2794 ts_tablet_manager.cc:1962] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:34.004002 2830 tablet.cc:1759] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20261118 14:58:33.962688 2809 raft_consensus.cc:386] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:34.009146 2818 tablet_bootstrap.cc:492] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:34.009507 2712 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" instance_seqno: 1795013887687621) as {username='slave'} at 127.0.72.129:41045; Asking this server to re-register.
I20261118 14:58:34.011439 984 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:34.013260 2712 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:34.014600 2818 tablet_bootstrap.cc:654] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:34.017084 2812 ts_tablet_manager.cc:1975] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:34.017179 2812 log.cc:1199] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/f6ce7737d98348659f0c83cb81f7ab45
I20261118 14:58:34.018419 2804 ts_tablet_manager.cc:1996] T ad6f4e435e914055a4ee7d5afa5e5f6a P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:34.019047 2829 tablet.cc:1759] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20261118 14:58:34.020555 2804 tablet_replica.cc:333] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:34.020787 2804 raft_consensus.cc:2261] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:34.020957 2804 raft_consensus.cc:2290] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:34.021508 2804 ts_tablet_manager.cc:1962] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:34.022867 2808 ts_tablet_manager.cc:1480] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.038s user 0.001s sys 0.002s
I20261118 14:58:34.033782 2808 tablet_bootstrap.cc:492] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:34.034847 2794 ts_tablet_manager.cc:1975] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:34.034966 2794 log.cc:1199] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/ad6f4e435e914055a4ee7d5afa5e5f6a
I20261118 14:58:34.035404 2794 ts_tablet_manager.cc:1996] T ad6f4e435e914055a4ee7d5afa5e5f6a P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:34.036593 2794 tablet_replica.cc:333] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:34.036895 2794 raft_consensus.cc:2261] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:34.037050 2794 raft_consensus.cc:2290] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:34.037647 2794 ts_tablet_manager.cc:1962] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:34.042431 2809 raft_consensus.cc:748] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:34.043285 2809 consensus_queue.cc:261] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.044894 2809 ts_tablet_manager.cc:1480] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.083s user 0.002s sys 0.000s
I20261118 14:58:34.045164 2809 tablet_bootstrap.cc:492] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:34.047128 2808 tablet_bootstrap.cc:654] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
W20261118 14:58:34.055871 1148 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.72.130:38257, user_credentials={real_user=slave}} blocked reactor thread for 51680.3us
I20261118 14:58:34.056990 2815 tablet.cc:1759] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20261118 14:58:34.057765 2809 tablet_bootstrap.cc:654] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
W20261118 14:58:34.064436 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:34.065850 2794 ts_tablet_manager.cc:1975] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:34.065972 2794 log.cc:1199] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:34.067627 2794 ts_tablet_manager.cc:1996] T a78ecb3a620242f4a16128b0d456c352 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:34.069223 2808 tablet_bootstrap.cc:492] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:34.069384 2808 ts_tablet_manager.cc:1449] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.000s
I20261118 14:58:34.070176 2808 raft_consensus.cc:360] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.070395 2808 raft_consensus.cc:386] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:34.070847 2808 raft_consensus.cc:748] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:34.071072 2808 consensus_queue.cc:261] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.071563 2808 ts_tablet_manager.cc:1480] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 14:58:34.072781 2812 ts_tablet_manager.cc:1996] T f6ce7737d98348659f0c83cb81f7ab45 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:34.073858 2818 tablet_bootstrap.cc:492] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:34.074011 2818 ts_tablet_manager.cc:1449] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.065s user 0.003s sys 0.000s
I20261118 14:58:34.075845 2812 tablet_replica.cc:333] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:34.076031 2812 raft_consensus.cc:2261] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:34.076202 2812 raft_consensus.cc:2290] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:34.076723 2812 ts_tablet_manager.cc:1962] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:34.078421 2804 ts_tablet_manager.cc:1975] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:34.078538 2804 log.cc:1199] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/f6ce7737d98348659f0c83cb81f7ab45
I20261118 14:58:34.078919 2804 ts_tablet_manager.cc:1996] T f6ce7737d98348659f0c83cb81f7ab45 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:34.080145 2818 raft_consensus.cc:360] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.080327 2818 raft_consensus.cc:386] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:34.080391 2818 raft_consensus.cc:748] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:34.080622 2818 consensus_queue.cc:261] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.082638 2809 tablet_bootstrap.cc:492] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:34.082975 2809 ts_tablet_manager.cc:1449] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.002s
I20261118 14:58:34.083781 2809 raft_consensus.cc:360] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.083953 2809 raft_consensus.cc:386] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:34.084008 2809 raft_consensus.cc:748] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:34.084219 2809 consensus_queue.cc:261] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.084414 2818 ts_tablet_manager.cc:1480] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20261118 14:58:34.085920 2794 tablet_replica.cc:333] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:34.086086 2794 raft_consensus.cc:2261] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:34.086272 2794 raft_consensus.cc:2290] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:34.086807 2794 ts_tablet_manager.cc:1962] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:34.086890 2809 ts_tablet_manager.cc:1480] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20261118 14:58:34.089432 2812 ts_tablet_manager.cc:1975] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:34.089551 2812 log.cc:1199] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/ad6f4e435e914055a4ee7d5afa5e5f6a
I20261118 14:58:34.090442 2812 ts_tablet_manager.cc:1996] T ad6f4e435e914055a4ee7d5afa5e5f6a P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:34.098582 2794 ts_tablet_manager.cc:1975] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:34.098707 2794 log.cc:1199] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/f6ce7737d98348659f0c83cb81f7ab45
I20261118 14:58:34.099073 2794 ts_tablet_manager.cc:1996] T f6ce7737d98348659f0c83cb81f7ab45 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:34.120579 2630 raft_consensus.cc:499] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:34.120783 2630 raft_consensus.cc:521] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.121420 2630 leader_election.cc:290] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:34.121954 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a8655d55cde2410184cf0c74683f8717" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:34.122088 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a8655d55cde2410184cf0c74683f8717" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:34.122172 1211 raft_consensus.cc:2490] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:34.122282 1075 raft_consensus.cc:2490] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:34.122704 874 leader_election.cc:304] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:34.123050 2630 raft_consensus.cc:2826] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:34.123139 2630 raft_consensus.cc:499] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:34.123246 2630 raft_consensus.cc:3096] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:34.125041 2630 raft_consensus.cc:521] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.125577 2630 leader_election.cc:290] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:34.126022 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a8655d55cde2410184cf0c74683f8717" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:34.126099 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a8655d55cde2410184cf0c74683f8717" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:34.126215 1075 raft_consensus.cc:3096] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:34.126256 1211 raft_consensus.cc:3096] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:34.129074 1211 raft_consensus.cc:2490] T a8655d55cde2410184cf0c74683f8717 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:34.129069 1075 raft_consensus.cc:2490] T a8655d55cde2410184cf0c74683f8717 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:34.129477 875 leader_election.cc:304] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:34.129781 2630 raft_consensus.cc:2826] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:34.130154 2630 raft_consensus.cc:705] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:34.130712 2630 consensus_queue.cc:238] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.134338 2840 consensus_queue.cc:1049] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:34.137621 2840 consensus_queue.cc:1049] T a8655d55cde2410184cf0c74683f8717 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 14:58:34.155089 875 consensus_peers.cc:597] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352. This is attempt 1: this message will repeat every 5th retry.
W20261118 14:58:34.210145 873 connection.cc:573] server connection from 127.0.0.1:37328 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:34.210134 1146 connection.cc:573] server connection from 127.0.0.1:38756 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:34.210505 1265 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:34.210561 1011 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:34.210975 1011 connection.cc:573] server connection from 127.0.0.1:45256 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:34.211323 1265 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.189:46707)
W20261118 14:58:34.211915 1265 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.189:46707)
W20261118 14:58:34.211915 873 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:34.212395 1265 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.189:46707)
W20261118 14:58:34.213035 1265 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.189:46707)
W20261118 14:58:34.214212 2788 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 14:58:34.211415 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.189:46707 (local address 127.0.72.130:45605)
1118 14:58:34.214084 (+ 2669us) negotiation.cc:107] Waiting for socket to connect
1118 14:58:34.214162 (+ 78us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":2540}
W20261118 14:58:34.215571 1146 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
I20261118 14:58:34.389395 2846 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:34.389642 2846 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:34.391422 2846 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:34.392319 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:34.393754 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:34.394062 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:34.394188 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:34.394556 2846 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20261118 14:58:34.435101 2653 raft_consensus.cc:499] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:34.435302 2653 raft_consensus.cc:521] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.436019 2653 leader_election.cc:290] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:34.436635 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9cc4915e3e0344f3b262ba85f16d9be4" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:34.436612 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9cc4915e3e0344f3b262ba85f16d9be4" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:34.436864 938 raft_consensus.cc:2490] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:34.436880 1075 raft_consensus.cc:2490] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:34.437373 1148 leader_election.cc:304] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:34.437665 2653 raft_consensus.cc:2826] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:34.437791 2653 raft_consensus.cc:499] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:34.437891 2653 raft_consensus.cc:3096] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:34.440315 2653 raft_consensus.cc:521] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.441264 2653 leader_election.cc:290] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:34.441489 938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9cc4915e3e0344f3b262ba85f16d9be4" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:34.441572 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9cc4915e3e0344f3b262ba85f16d9be4" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:34.441658 938 raft_consensus.cc:3096] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:34.441766 1075 raft_consensus.cc:3096] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:34.443688 938 raft_consensus.cc:2490] T 9cc4915e3e0344f3b262ba85f16d9be4 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:34.443796 1075 raft_consensus.cc:2490] T 9cc4915e3e0344f3b262ba85f16d9be4 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:34.444111 1145 leader_election.cc:304] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:34.444355 2653 raft_consensus.cc:2826] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:34.444583 2653 raft_consensus.cc:705] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:34.444942 2653 consensus_queue.cc:238] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:34.447615 2831 consensus_queue.cc:1049] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:34.449015 2831 consensus_queue.cc:1049] T 9cc4915e3e0344f3b262ba85f16d9be4 P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 14:58:34.484588 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:34.930546 2855 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:34.930761 2855 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:34.931542 2855 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:34.933631 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:34.933809 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:34.933951 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:34.934188 2855 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20261118 14:58:35.869910 2872 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:35.870093 2872 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:35.870805 2872 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:35.872690 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:35.872838 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:35.872988 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:35.873235 2872 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20261118 14:58:36.813603 290 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
/tmp/dist-test-taskyLAIFs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:46707
--webserver_interface=127.0.72.189
--webserver_port=42561
--builtin_ntp_servers=127.0.72.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 14:58:37.010206 2895 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:37.010421 2895 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:37.011101 2895 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:37.013126 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:37.013761 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:37.013883 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:37.014106 2895 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20261118 14:58:37.144483 2892 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 14:58:37.144778 2892 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 14:58:37.144884 2892 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 14:58:37.153131 2892 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 14:58:37.153232 2892 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 14:58:37.153321 2892 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 14:58:37.153381 2892 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 14:58:37.163534 2892 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:40455
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:46707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=42561
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 14:43:24 UTC on bdcb31816ec0
build id 14845
ASAN enabled
I20261118 14:58:37.165292 2892 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 14:58:37.167207 2892 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 14:58:37.177706 2908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:37.177829 2909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 14:58:37.179100 2911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 14:58:37.179647 2892 server_base.cc:1061] running on GCE node
I20261118 14:58:37.180457 2892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 14:58:37.181860 2892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 14:58:37.183039 2892 hybrid_clock.cc:648] HybridClock initialized: now 1795013917182985 us; error 43 us; skew 500 ppm
I20261118 14:58:37.183367 2892 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 14:58:37.185441 2892 webserver.cc:577] Webserver started at http://127.0.72.189:42561/ using document root <none> and password file <none>
I20261118 14:58:37.186157 2892 fs_manager.cc:362] Metadata directory not provided
I20261118 14:58:37.186245 2892 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 14:58:37.191717 2892 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20261118 14:58:37.194689 2917 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20261118 14:58:37.196282 2892 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20261118 14:58:37.196446 2892 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data,/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
uuid: "7c9ef96ff074450a879e66787486386d"
format_stamp: "Formatted at 2026-11-18 14:58:06 on dist-test-slave-qxzn"
I20261118 14:58:37.197144 2892 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 14:58:37.215077 2892 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20261118 14:58:37.216117 2892 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 14:58:37.216380 2892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 14:58:37.235034 2892 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:46707
I20261118 14:58:37.235081 2968 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:46707 every 8 connection(s)
I20261118 14:58:37.236523 2892 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/master-1/data/info.pb
I20261118 14:58:37.241200 290 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskyLAIFs/build/asan/bin/kudu as pid 2892
I20261118 14:58:37.243731 2969 sys_catalog.cc:263] Verifying existing consensus state
I20261118 14:58:37.247135 2969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap starting.
I20261118 14:58:37.266093 2969 log.cc:826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Log is configured to *not* fsync() on all Append() calls
I20261118 14:58:37.322986 986 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:37.355444 2969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap replayed 1/1 log segments. Stats: ops{read=121 overwritten=0 applied=121 ignored=0} inserts{seen=76 ignored=0} mutations{seen=115 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 14:58:37.355957 2969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Bootstrap complete.
I20261118 14:58:37.361095 2969 raft_consensus.cc:360] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:37.361702 2969 raft_consensus.cc:748] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c9ef96ff074450a879e66787486386d, State: Initialized, Role: FOLLOWER
I20261118 14:58:37.362272 2969 consensus_queue.cc:261] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 121, Last appended: 8.121, Last appended by leader: 121, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:37.363051 2974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:37.363322 2974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:37.364022 2969 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: configured and running, proceeding with master startup.
I20261118 14:58:37.373924 2892 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:58:37.379962 2892 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:37.381986 2988 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:37.382066 2988 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: loading cluster ID for follower catalog manager: success
I20261118 14:58:37.387458 2988 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: acquiring CA information for follower catalog manager: success
I20261118 14:58:37.390103 2988 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 14:58:37.474457 1124 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:37.476948 2934 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" instance_seqno: 1795013888201158) as {username='slave'} at 127.0.72.130:56887; Asking this server to re-register.
I20261118 14:58:37.477692 1124 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:37.479588 2934 ts_manager.cc:210] Registered new tserver with Master: 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
I20261118 14:58:37.505093 1259 heartbeater.cc:345] Connected to a master server at 127.0.72.189:46707
I20261118 14:58:37.506763 2934 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" instance_seqno: 1795013888668865) as {username='slave'} at 127.0.72.131:49137; Asking this server to re-register.
I20261118 14:58:37.507301 1259 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:37.508491 2934 ts_manager.cc:210] Registered new tserver with Master: c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:37.785120 2974 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:37.785444 2974 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:37.787937 2974 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
W20261118 14:58:37.788537 2918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:37.790093 2918 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:37.794938 2722 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 9 candidate_status { last_received { term: 8 index: 121 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" is_pre_election: true
I20261118 14:58:37.795121 2722 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c9ef96ff074450a879e66787486386d in term 8.
I20261118 14:58:37.795738 2920 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e
I20261118 14:58:37.796095 2974 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20261118 14:58:37.796250 2974 raft_consensus.cc:499] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:37.796378 2974 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 8 FOLLOWER]: Advancing to term 9
I20261118 14:58:37.800053 2974 raft_consensus.cc:521] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:37.800706 2974 leader_election.cc:290] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 9 election: Requested vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 9b90720bc0b34e4994abf4fc6f9e59e1 (127.0.72.188:33967)
I20261118 14:58:37.801331 2722 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c9ef96ff074450a879e66787486386d" candidate_term: 9 candidate_status { last_received { term: 8 index: 121 } } ignore_live_leader: false dest_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1"
I20261118 14:58:37.801635 2722 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 8 FOLLOWER]: Advancing to term 9
W20261118 14:58:37.802181 2918 leader_election.cc:336] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:37.803876 2722 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c9ef96ff074450a879e66787486386d in term 9.
I20261118 14:58:37.804299 2920 leader_election.cc:304] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c9ef96ff074450a879e66787486386d, 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e
I20261118 14:58:37.804556 2974 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 9 FOLLOWER]: Leader election won for term 9
I20261118 14:58:37.805022 2974 raft_consensus.cc:705] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [term 9 LEADER]: Becoming Leader. State: Replica: 7c9ef96ff074450a879e66787486386d, State: Running, Role: LEADER
I20261118 14:58:37.805495 2974 consensus_queue.cc:238] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 121, Committed index: 121, Last appended: 8.121, Last appended by leader: 121, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:37.808640 2997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c9ef96ff074450a879e66787486386d. Latest consensus state: current_term: 9 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:37.809072 2997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:37.809782 2722 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Refusing update from remote peer 7c9ef96ff074450a879e66787486386d: Log matching property violated. Preceding OpId in replica: term: 8 index: 121. Preceding OpId from leader: term: 9 index: 122. (index mismatch)
I20261118 14:58:37.810204 3001 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 14:58:37.810834 2997 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 122, Last known committed idx: 121, Time since last communication: 0.000s
W20261118 14:58:37.811335 2918 consensus_peers.cc:597] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d -> Peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Couldn't send request to peer 0ed65a3719634e519d783540afd6189e. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 14:58:37.813506 3002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c9ef96ff074450a879e66787486386d. Latest consensus state: current_term: 9 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:37.813750 3002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:37.814395 3001 catalog_manager.cc:694] Loaded metadata for table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]
I20261118 14:58:37.816044 3000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:37.816306 3000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d [sys.catalog]: This master's current role is: LEADER
I20261118 14:58:37.817382 3001 catalog_manager.cc:694] Loaded metadata for table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]
I20261118 14:58:37.818033 3001 catalog_manager.cc:694] Loaded metadata for table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0]
I20261118 14:58:37.818503 3001 catalog_manager.cc:694] Loaded metadata for table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627]
I20261118 14:58:37.819159 3001 catalog_manager.cc:694] Loaded metadata for table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]
I20261118 14:58:37.819613 3001 catalog_manager.cc:694] Loaded metadata for table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15]
I20261118 14:58:37.820245 3001 catalog_manager.cc:694] Loaded metadata for table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f]
I20261118 14:58:37.820665 3001 catalog_manager.cc:694] Loaded metadata for table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8]
I20261118 14:58:37.821055 3001 catalog_manager.cc:694] Loaded metadata for table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]
I20261118 14:58:37.821442 3001 catalog_manager.cc:694] Loaded metadata for table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b]
I20261118 14:58:37.822028 3001 catalog_manager.cc:694] Loaded metadata for table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7]
I20261118 14:58:37.826925 3002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "7c9ef96ff074450a879e66787486386d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } } }
I20261118 14:58:37.827379 3002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 14:58:37.827379 2934 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" instance_seqno: 1795013887687621) as {username='slave'} at 127.0.72.129:56953; Asking this server to re-register.
I20261118 14:58:37.827639 3001 tablet_loader.cc:96] loaded metadata for tablet 2c20495505a1478398a9c447cf18742b (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28])
I20261118 14:58:37.828161 986 heartbeater.cc:462] Registering TS with master...
I20261118 14:58:37.828213 3001 tablet_loader.cc:96] loaded metadata for tablet 42d8a3bab2694c3e83da6a8642d6d911 (table default.table_802ca0c5b2f34d6ab88c04070b3a28d1 [id=90670879a67746e1bb5410ceb11fcb15])
I20261118 14:58:37.828805 3001 tablet_loader.cc:96] loaded metadata for tablet 58209714937148ad8a3722b5e392bfce (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d])
I20261118 14:58:37.829093 3001 tablet_loader.cc:96] loaded metadata for tablet 6a249f4f523e414caa8a762c839e9129 (table default.table_023fd7c6f2b3407da4cbd94af99cda3a [id=ab183dcbe03044f79945c2707722c91f])
I20261118 14:58:37.829380 2934 ts_manager.cc:210] Registered new tserver with Master: 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:37.829385 3001 tablet_loader.cc:96] loaded metadata for tablet 6ffda5e2164e4a558231db3043422159 (table default.table_3f41e3f42a794945838124ce6712ceb3 [id=51a7f645ca6b4e56affbfecd1f1187a0])
I20261118 14:58:37.829841 3001 tablet_loader.cc:96] loaded metadata for tablet 77e643a760294dacbfca6305f1811bbd (table default.table_7171b2f0b6ea470393bf9aa28246cf65 [id=cdc2cda186804cd39b0d320f08e91df8])
I20261118 14:58:37.831156 3001 tablet_loader.cc:96] loaded metadata for tablet 9cc4915e3e0344f3b262ba85f16d9be4 (table default.table_a0ea13e08f4e45bc84d5936c615c8ff7 [id=69ddcff335ba4d85bba72a380a850627])
I20261118 14:58:37.831817 3001 tablet_loader.cc:96] loaded metadata for tablet a8655d55cde2410184cf0c74683f8717 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8])
I20261118 14:58:37.832942 3001 tablet_loader.cc:96] loaded metadata for tablet d5ebf8d258a644d9bea96b42357a9345 (table default.table_2f1acae65de24ae9a7a19515d10acf9c [id=e26c2363b15444a5882d6ccf7e290f8b])
I20261118 14:58:37.833221 3001 tablet_loader.cc:96] loaded metadata for tablet da163bd4e25b4254ab3923af361202ce (table default.table_927d6a72d3524190964efb6e9f1f8d42 [id=fae4a508ca0b4035a4843cdcfa56f4e7])
I20261118 14:58:37.833634 3001 tablet_loader.cc:96] loaded metadata for tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6])
I20261118 14:58:37.834687 3001 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 14:58:37.836014 3001 catalog_manager.cc:1310] Loaded cluster ID: 93ce58d3ae0d4f0595d7f8a23cb2bc15
I20261118 14:58:37.836086 3001 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 14:58:37.837679 3001 catalog_manager.cc:1555] Loading token signing keys...
I20261118 14:58:37.838933 3001 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Loaded TSK: 0
I20261118 14:58:37.839788 3001 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 14:58:37.840957 3001 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 14:58:37.842175 3001 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 33
I20261118 14:58:37.881125 918 tablet_service.cc:1547] Processing DeleteTablet for tablet a78ecb3a620242f4a16128b0d456c352 with delete_type TABLET_DATA_DELETED (replaced by tablet f6ce7737d98348659f0c83cb81f7ab45) from {username='slave'} at 127.0.0.1:37332
I20261118 14:58:37.881654 3009 tablet_replica.cc:333] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:37.881829 3009 raft_consensus.cc:2261] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:37.882164 3009 raft_consensus.cc:2290] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.882550 2934 master_service.cc:965] ReplaceTablet: received request to replace tablet 58209714937148ad8a3722b5e392bfce from {username='slave'} at 127.0.0.1:38946
I20261118 14:58:37.882896 3009 ts_tablet_manager.cc:1962] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:37.884004 2925 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9" } modify_external_catalogs: true
I20261118 14:58:37.884397 2925 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_17fd908a9ffe47a888a241bdbf0d9ff9" } modify_external_catalogs: true
I20261118 14:58:37.885802 2930 master_service.cc:965] ReplaceTablet: received request to replace tablet 2c20495505a1478398a9c447cf18742b from {username='slave'} at 127.0.0.1:38946
I20261118 14:58:37.885704 2932 master_service.cc:965] ReplaceTablet: received request to replace tablet 2c20495505a1478398a9c447cf18742b from {username='slave'} at 127.0.0.1:38946
I20261118 14:58:37.883908 2926 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38946:
name: "default.Table_4dba69207cca45529e828162200c13f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:37.886232 2928 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38946:
name: "default.Table_d24daba84f8c4c94824da319564f0453"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:37.887895 2929 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38946:
name: "default.Table_4789eba8c66842f6ae326364062589fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:37.886700 2927 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38946:
name: "default.Table_053874e1edfe4b8eac52b4ba5f1b1870"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20261118 14:58:37.889065 3009 ts_tablet_manager.cc:1975] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:37.889182 3009 log.cc:1199] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:37.888957 2931 master_service.cc:965] ReplaceTablet: received request to replace tablet dd0cddf6622a469eb598e9740407a9f0 from {username='slave'} at 127.0.0.1:38946
I20261118 14:58:37.889605 3009 ts_tablet_manager.cc:1996] T a78ecb3a620242f4a16128b0d456c352 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:37.889012 2933 master_service.cc:965] ReplaceTablet: received request to replace tablet f6ce7737d98348659f0c83cb81f7ab45 from {username='slave'} at 127.0.0.1:38946
I20261118 14:58:37.890821 2918 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet a78ecb3a620242f4a16128b0d456c352 (table default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af [id=d65d9ba97cda44e3bdff3add1fec1ec8]) successfully deleted
F20261118 14:58:37.891073 2987 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 36, previous 36; dumping first 4 out of 5 received events: NotificationEvent(eventId=34, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_4789eba8c66842f6ae326364062589fb, message={"server":"","servicePrincipal":"","db":"default","table":"table_4789eba8c66842f6ae326364062589fb","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4789eba8c66842f6ae326364062589fb\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4789eba8c66842f6ae326364062589fb\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4789eba8c66842f6ae326364062589fb\",\"kudu.table_id\":\"0ce3ba58cd7440a9b27427e7713af331\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=35, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_7171b2f0b6ea470393bf9aa28246cf65, message={"server":"","servicePrincipal":"","db":"default","table":"table_7171b2f0b6ea470393bf9aa28246cf65","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_7171b2f0b6ea470393bf9aa28246cf65\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_7171b2f0b6ea470393bf9aa28246cf65\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_7171b2f0b6ea470393bf9aa28246cf65\",\"kudu.table_id\":\"cdc2cda186804cd39b0d320f08e91df8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=CREATE_TABLE, dbName=default, tableName=table_4d2c8b2bcb994782a8bbf4b38ca2f7af, message={"server":"","servicePrincipal":"","db":"default","table":"table_4d2c8b2bcb994782a8bbf4b38ca2f7af","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_4d2c8b2bcb994782a8bbf4b38ca2f7af\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_4d2c8b2bcb994782a8bbf4b38ca2f7af\",\"kudu.table_id\":\"d65d9ba97cda44e3bdff3add1fec1ec8\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=36, eventTime=1795013896, eventType=DROP_TABLE, dbName=default, tableName=table_36b9017c9ae24ecdb8ae81bf85341052, message={"server":"","servicePrincipal":"","db":"default","table":"table_36b9017c9ae24ecdb8ae81bf85341052","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_36b9017c9ae24ecdb8ae81bf85341052\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1795013896},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/warehouse/table_36b9017c9ae24ecdb8ae81bf85341052\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_36b9017c9ae24ecdb8ae81bf85341052\",\"kudu.table_id\":\"378bd168f216477992c1b7bb33791778\",\"transient_lastDdlTime\":\"1795013896\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"93ce58d3ae0d4f0595d7f8a23cb2bc15\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1795013896}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20261118 14:58:37.891280 2933 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38946:
name: "default.Table_5c2ab051f54546cab77c75d60b2a8bd1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
@ 0x7f281fbced5d google::LogMessage::Fail() at ??:0
@ 0x7f281fbd2b23 google::LogMessage::SendToLog() at ??:0
@ 0x7f281fbce75c google::LogMessage::Flush() at ??:0
@ 0x7f281fbce9a1 google::LogMessage::~LogMessage() at ??:0
I20261118 14:58:37.904711 2931 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet dd0cddf6622a469eb598e9740407a9f0
I20261118 14:58:37.904932 2934 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet 58209714937148ad8a3722b5e392bfce
@ 0x7f283504e9a4 kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 14:58:37.907050 918 tablet_service.cc:1547] Processing DeleteTablet for tablet dd0cddf6622a469eb598e9740407a9f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e1c16e124e54a9e934a3af82ff8f5a0) from {username='slave'} at 127.0.0.1:37332
I20261118 14:58:37.907298 3009 tablet_replica.cc:333] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:37.907382 919 tablet_service.cc:1547] Processing DeleteTablet for tablet 58209714937148ad8a3722b5e392bfce with delete_type TABLET_DATA_DELETED (replaced by tablet 055fee087c2d4bd0a15b1b70d131d78d) from {username='slave'} at 127.0.0.1:37332
I20261118 14:58:37.907500 3009 raft_consensus.cc:2261] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:37.907685 3009 raft_consensus.cc:2290] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.908130 2931 catalog_manager.cc:6936] ReplaceTablet: tablet dd0cddf6622a469eb598e9740407a9f0 deleted and replaced by tablet 1e1c16e124e54a9e934a3af82ff8f5a0
I20261118 14:58:37.908244 3009 ts_tablet_manager.cc:1962] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:37.908435 2931 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1" } modify_external_catalogs: true
I20261118 14:58:37.908638 2931 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_802ca0c5b2f34d6ab88c04070b3a28d1" } modify_external_catalogs: true
I20261118 14:58:37.909272 2934 catalog_manager.cc:6936] ReplaceTablet: tablet 58209714937148ad8a3722b5e392bfce deleted and replaced by tablet 055fee087c2d4bd0a15b1b70d131d78d
I20261118 14:58:37.909591 2934 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" } modify_external_catalogs: true
I20261118 14:58:37.909813 2934 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_277b8485c5ba4cdfaa228990cb8a41ed" } modify_external_catalogs: true
I20261118 14:58:37.915120 3009 ts_tablet_manager.cc:1975] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:37.915252 3009 log.cc:1199] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/dd0cddf6622a469eb598e9740407a9f0
I20261118 14:58:37.915632 3009 ts_tablet_manager.cc:1996] T dd0cddf6622a469eb598e9740407a9f0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:37.915985 2930 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 7c9ef96ff074450a879e66787486386d: Sending DeleteTablet for 3 replicas of tablet 2c20495505a1478398a9c447cf18742b
I20261118 14:58:37.916495 2930 catalog_manager.cc:6936] ReplaceTablet: tablet 2c20495505a1478398a9c447cf18742b deleted and replaced by tablet 3686f13df75044e3817bed5606bcc618
I20261118 14:58:37.917016 919 tablet_service.cc:1547] Processing DeleteTablet for tablet 2c20495505a1478398a9c447cf18742b with delete_type TABLET_DATA_DELETED (replaced by tablet 3686f13df75044e3817bed5606bcc618) from {username='slave'} at 127.0.0.1:37332
I20261118 14:58:37.918483 2918 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]) successfully deleted
I20261118 14:58:37.918784 3009 tablet_replica.cc:333] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:37.918952 3009 raft_consensus.cc:2261] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:37.919129 3009 raft_consensus.cc:2290] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
@ 0x7f283504d251 kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 14:58:37.919633 3009 ts_tablet_manager.cc:1962] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f2835054ecc kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 14:58:37.925933 1055 tablet_service.cc:1547] Processing DeleteTablet for tablet dd0cddf6622a469eb598e9740407a9f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e1c16e124e54a9e934a3af82ff8f5a0) from {username='slave'} at 127.0.0.1:45270
I20261118 14:58:37.926312 1057 tablet_service.cc:1547] Processing DeleteTablet for tablet 58209714937148ad8a3722b5e392bfce with delete_type TABLET_DATA_DELETED (replaced by tablet 055fee087c2d4bd0a15b1b70d131d78d) from {username='slave'} at 127.0.0.1:45270
I20261118 14:58:37.926559 1053 tablet_service.cc:1547] Processing DeleteTablet for tablet 2c20495505a1478398a9c447cf18742b with delete_type TABLET_DATA_DELETED (replaced by tablet 3686f13df75044e3817bed5606bcc618) from {username='slave'} at 127.0.0.1:45270
I20261118 14:58:37.926506 2930 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_a0ea13e08f4e45bc84d5936c615c8ff7" } new_table_name: "default.Table_8afe8b6b4fe4433882ded7831ce722e9" modify_external_catalogs: true
I20261118 14:58:37.926602 3018 tablet_replica.cc:333] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:37.926779 3018 raft_consensus.cc:2261] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:37.926831 3009 ts_tablet_manager.cc:1975] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:37.926919 3009 log.cc:1199] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/58209714937148ad8a3722b5e392bfce
I20261118 14:58:37.927132 3018 raft_consensus.cc:2290] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.927285 3009 ts_tablet_manager.cc:1996] T 58209714937148ad8a3722b5e392bfce P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:37.927659 3018 ts_tablet_manager.cc:1962] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:37.927706 2932 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:37.927901 2932 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38946:
table { table_name: "default.Table_023fd7c6f2b3407da4cbd94af99cda3a" } modify_external_catalogs: true
I20261118 14:58:37.928232 2918 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 58209714937148ad8a3722b5e392bfce (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
@ 0x7f2835054d49 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x372c7d std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 14:58:37.930680 3009 tablet_replica.cc:333] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: stopping tablet replica
I20261118 14:58:37.930852 3009 raft_consensus.cc:2261] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:37.931003 3009 raft_consensus.cc:2290] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.931478 3009 ts_tablet_manager.cc:1962] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting tablet data with delete state TABLET_DATA_DELETED
@ 0x7f2821017dc9 kudu::Thread::SuperviseThread() at ??:0
I20261118 14:58:37.934892 1188 tablet_service.cc:1547] Processing DeleteTablet for tablet 58209714937148ad8a3722b5e392bfce with delete_type TABLET_DATA_DELETED (replaced by tablet 055fee087c2d4bd0a15b1b70d131d78d) from {username='slave'} at 127.0.0.1:38770
I20261118 14:58:37.935144 1189 tablet_service.cc:1547] Processing DeleteTablet for tablet dd0cddf6622a469eb598e9740407a9f0 with delete_type TABLET_DATA_DELETED (replaced by tablet 1e1c16e124e54a9e934a3af82ff8f5a0) from {username='slave'} at 127.0.0.1:38770
I20261118 14:58:37.935374 1187 tablet_service.cc:1547] Processing DeleteTablet for tablet 2c20495505a1478398a9c447cf18742b with delete_type TABLET_DATA_DELETED (replaced by tablet 3686f13df75044e3817bed5606bcc618) from {username='slave'} at 127.0.0.1:38770
I20261118 14:58:37.938426 3009 ts_tablet_manager.cc:1975] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:37.938546 3009 log.cc:1199] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-0/wal/wals/2c20495505a1478398a9c447cf18742b
I20261118 14:58:37.938715 3019 tablet_replica.cc:333] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:37.938901 3009 ts_tablet_manager.cc:1996] T 2c20495505a1478398a9c447cf18742b P 7c53fe91804746e8bfcf743e0ab8ba6f: Deleting consensus metadata
I20261118 14:58:37.938897 3019 raft_consensus.cc:2261] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:37.939247 3019 raft_consensus.cc:2290] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.939222 1055 tablet_service.cc:1500] Processing CreateTablet for tablet 1e1c16e124e54a9e934a3af82ff8f5a0 (DEFAULT_TABLE table=default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.939512 919 tablet_service.cc:1500] Processing CreateTablet for tablet 1e1c16e124e54a9e934a3af82ff8f5a0 (DEFAULT_TABLE table=default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.939625 1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1c16e124e54a9e934a3af82ff8f5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.939823 1187 tablet_service.cc:1500] Processing CreateTablet for tablet 055fee087c2d4bd0a15b1b70d131d78d (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.939914 919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1c16e124e54a9e934a3af82ff8f5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.940207 1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055fee087c2d4bd0a15b1b70d131d78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.941730 3018 ts_tablet_manager.cc:1975] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:37.941841 3018 log.cc:1199] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/dd0cddf6622a469eb598e9740407a9f0
I20261118 14:58:37.942232 3018 ts_tablet_manager.cc:1996] T dd0cddf6622a469eb598e9740407a9f0 P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:37.943320 2921 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet dd0cddf6622a469eb598e9740407a9f0 (table default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]) successfully deleted
I20261118 14:58:37.943608 3018 tablet_replica.cc:333] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:37.943815 3018 raft_consensus.cc:2261] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:37.943984 3018 raft_consensus.cc:2290] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.944070 1053 tablet_service.cc:1500] Processing CreateTablet for tablet 055fee087c2d4bd0a15b1b70d131d78d (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.944418 1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055fee087c2d4bd0a15b1b70d131d78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.944552 3018 ts_tablet_manager.cc:1962] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:37.945690 918 tablet_service.cc:1500] Processing CreateTablet for tablet 055fee087c2d4bd0a15b1b70d131d78d (DEFAULT_TABLE table=default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.946020 918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 055fee087c2d4bd0a15b1b70d131d78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.948639 1057 tablet_service.cc:1500] Processing CreateTablet for tablet 3686f13df75044e3817bed5606bcc618 (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.948714 915 tablet_service.cc:1500] Processing CreateTablet for tablet 3686f13df75044e3817bed5606bcc618 (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.949002 1057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3686f13df75044e3817bed5606bcc618. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.949024 915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3686f13df75044e3817bed5606bcc618. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.949159 1189 tablet_service.cc:1500] Processing CreateTablet for tablet 3686f13df75044e3817bed5606bcc618 (DEFAULT_TABLE table=default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.949504 1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3686f13df75044e3817bed5606bcc618. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.951331 3018 ts_tablet_manager.cc:1975] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:37.951442 3018 log.cc:1199] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/58209714937148ad8a3722b5e392bfce
I20261118 14:58:37.951783 3018 ts_tablet_manager.cc:1996] T 58209714937148ad8a3722b5e392bfce P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:37.952687 2921 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 58209714937148ad8a3722b5e392bfce (table default.table_277b8485c5ba4cdfaa228990cb8a41ed [id=07fa88e7fcd044fca002cc02a8788a4d]) successfully deleted
I20261118 14:58:37.953001 3018 tablet_replica.cc:333] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: stopping tablet replica
I20261118 14:58:37.953176 3018 raft_consensus.cc:2261] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Raft consensus shutting down.
I20261118 14:58:37.953483 3018 raft_consensus.cc:2290] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.954632 3018 ts_tablet_manager.cc:1962] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:37.954655 1188 tablet_service.cc:1500] Processing CreateTablet for tablet 1e1c16e124e54a9e934a3af82ff8f5a0 (DEFAULT_TABLE table=default.table_17fd908a9ffe47a888a241bdbf0d9ff9 [id=4c79d8bd953e4723bb1875c985977fc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 14:58:37.957068 1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e1c16e124e54a9e934a3af82ff8f5a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 14:58:37.959164 2918 catalog_manager.cc:5065] TS 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): tablet 2c20495505a1478398a9c447cf18742b (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]) successfully deleted
I20261118 14:58:37.959452 3019 ts_tablet_manager.cc:1962] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:37.960655 3018 ts_tablet_manager.cc:1975] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:37.960770 3018 log.cc:1199] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-1/wal/wals/2c20495505a1478398a9c447cf18742b
I20261118 14:58:37.961199 3018 ts_tablet_manager.cc:1996] T 2c20495505a1478398a9c447cf18742b P 245c4fe5e44448d98db1eed6329d150d: Deleting consensus metadata
I20261118 14:58:37.962143 2921 catalog_manager.cc:5065] TS 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): tablet 2c20495505a1478398a9c447cf18742b (table default.table_a69dc21b9f4948ffad3423d68bff8f34 [id=886aa9bb52444e1186957fd7e610fb28]) successfully deleted
I20261118 14:58:37.962690 3023 tablet_bootstrap.cc:492] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:37.963217 3021 tablet_bootstrap.cc:492] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:37.964376 3023 tablet_bootstrap.cc:654] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:37.965427 3022 tablet_bootstrap.cc:492] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
@ 0x7f28246f26db start_thread at ??:0
I20261118 14:58:37.966951 3021 tablet_bootstrap.cc:654] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:37.969981 3019 ts_tablet_manager.cc:1975] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:37.970089 3019 log.cc:1199] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/58209714937148ad8a3722b5e392bfce
@ 0x7f281d16261f clone at ??:0
I20261118 14:58:37.970782 3019 ts_tablet_manager.cc:1996] T 58209714937148ad8a3722b5e392bfce P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:37.971696 3022 tablet_bootstrap.cc:654] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:37.973744 3023 tablet_bootstrap.cc:492] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:37.973871 3023 ts_tablet_manager.cc:1449] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20261118 14:58:37.974134 3019 tablet_replica.cc:333] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:37.974321 3019 raft_consensus.cc:2261] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:37.974510 3019 raft_consensus.cc:2290] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.974635 3023 raft_consensus.cc:360] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:37.974800 3023 raft_consensus.cc:386] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:37.974881 3023 raft_consensus.cc:748] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:37.975039 3019 ts_tablet_manager.cc:1962] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:37.975114 3023 consensus_queue.cc:261] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:37.975674 3023 ts_tablet_manager.cc:1480] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:37.975947 3023 tablet_bootstrap.cc:492] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:37.977695 3023 tablet_bootstrap.cc:654] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:37.981069 3019 ts_tablet_manager.cc:1975] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 14:58:37.981189 3019 log.cc:1199] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/dd0cddf6622a469eb598e9740407a9f0
I20261118 14:58:37.981544 3019 ts_tablet_manager.cc:1996] T dd0cddf6622a469eb598e9740407a9f0 P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:37.983783 3021 tablet_bootstrap.cc:492] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:37.983928 3021 ts_tablet_manager.cc:1449] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.004s
I20261118 14:58:37.984719 3021 raft_consensus.cc:360] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:37.984903 3021 raft_consensus.cc:386] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:37.984984 3021 raft_consensus.cc:748] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:37.985226 3021 consensus_queue.cc:261] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:37.986265 986 heartbeater.cc:500] Master 127.0.72.189:46707 was elected leader, sending a full tablet report...
I20261118 14:58:37.988101 3021 ts_tablet_manager.cc:1480] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20261118 14:58:37.988597 3021 tablet_bootstrap.cc:492] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:37.990585 3021 tablet_bootstrap.cc:654] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:37.991647 3019 tablet_replica.cc:333] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: stopping tablet replica
I20261118 14:58:37.991832 3019 raft_consensus.cc:2261] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 14:58:37.992053 3019 raft_consensus.cc:2290] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 14:58:37.992624 3019 ts_tablet_manager.cc:1962] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 14:58:37.999856 3021 tablet_bootstrap.cc:492] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:38.000026 3021 ts_tablet_manager.cc:1449] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.003s
I20261118 14:58:38.001240 3021 raft_consensus.cc:360] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.001343 3022 tablet_bootstrap.cc:492] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:38.001437 3021 raft_consensus.cc:386] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:38.001482 3022 ts_tablet_manager.cc:1449] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.036s user 0.008s sys 0.008s
I20261118 14:58:38.001510 3021 raft_consensus.cc:748] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:38.001798 3021 consensus_queue.cc:261] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.002326 3021 ts_tablet_manager.cc:1480] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20261118 14:58:38.003751 3019 ts_tablet_manager.cc:1975] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 14:58:38.003863 3019 log.cc:1199] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: Deleting WAL directory at /tmp/dist-test-taskyLAIFs/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1795013870656599-290-0/minicluster-data/ts-2/wal/wals/2c20495505a1478398a9c447cf18742b
I20261118 14:58:38.003957 3021 tablet_bootstrap.cc:492] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f: Bootstrap starting.
I20261118 14:58:38.004262 3019 ts_tablet_manager.cc:1996] T 2c20495505a1478398a9c447cf18742b P c01f1b88968f4905ba5a401a81ade9f7: Deleting consensus metadata
I20261118 14:58:38.005439 3022 raft_consensus.cc:360] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.005605 3022 raft_consensus.cc:386] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:38.005684 3022 raft_consensus.cc:748] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:38.005744 3021 tablet_bootstrap.cc:654] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:38.006157 3022 consensus_queue.cc:261] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.006646 3023 tablet_bootstrap.cc:492] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:38.006815 3023 ts_tablet_manager.cc:1449] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.002s
I20261118 14:58:38.007537 3023 raft_consensus.cc:360] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:38.007740 3023 raft_consensus.cc:386] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:38.007799 3023 raft_consensus.cc:748] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:38.008024 3023 consensus_queue.cc:261] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:38.008512 3023 ts_tablet_manager.cc:1480] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:38.008839 3023 tablet_bootstrap.cc:492] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d: Bootstrap starting.
I20261118 14:58:38.008857 3022 ts_tablet_manager.cc:1480] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20261118 14:58:38.009933 3022 tablet_bootstrap.cc:492] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:38.011596 3022 tablet_bootstrap.cc:654] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:38.013240 3022 tablet_bootstrap.cc:492] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:38.013379 3022 ts_tablet_manager.cc:1449] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20261118 14:58:38.014138 3022 raft_consensus.cc:360] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.014312 3022 raft_consensus.cc:386] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:38.014410 3022 raft_consensus.cc:748] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:38.014633 3022 consensus_queue.cc:261] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.015077 3022 ts_tablet_manager.cc:1480] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:38.015337 3022 tablet_bootstrap.cc:492] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7: Bootstrap starting.
I20261118 14:58:38.016284 3023 tablet_bootstrap.cc:654] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:38.016928 3022 tablet_bootstrap.cc:654] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7: Neither blocks nor log segments found. Creating new log.
I20261118 14:58:38.019335 3022 tablet_bootstrap.cc:492] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7: No bootstrap required, opened a new log
I20261118 14:58:38.019454 3022 ts_tablet_manager.cc:1449] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20261118 14:58:38.020118 3022 raft_consensus.cc:360] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:38.020275 3022 raft_consensus.cc:386] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:38.020352 3022 raft_consensus.cc:748] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Initialized, Role: FOLLOWER
I20261118 14:58:38.020586 3022 consensus_queue.cc:261] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:38.022037 3022 ts_tablet_manager.cc:1480] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20261118 14:58:38.023025 3021 tablet_bootstrap.cc:492] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f: No bootstrap required, opened a new log
I20261118 14:58:38.023187 3021 ts_tablet_manager.cc:1449] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.001s
I20261118 14:58:38.023840 3021 raft_consensus.cc:360] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.024000 3021 raft_consensus.cc:386] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:38.024066 3021 raft_consensus.cc:748] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Initialized, Role: FOLLOWER
I20261118 14:58:38.024122 3023 tablet_bootstrap.cc:492] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d: No bootstrap required, opened a new log
I20261118 14:58:38.024255 3023 ts_tablet_manager.cc:1449] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.003s
I20261118 14:58:38.024266 3021 consensus_queue.cc:261] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.024765 3021 ts_tablet_manager.cc:1480] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20261118 14:58:38.024950 3023 raft_consensus.cc:360] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.025115 3023 raft_consensus.cc:386] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 14:58:38.025188 3023 raft_consensus.cc:748] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Initialized, Role: FOLLOWER
I20261118 14:58:38.025399 3023 consensus_queue.cc:261] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.025941 3023 ts_tablet_manager.cc:1480] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20261118 14:58:38.146397 3007 raft_consensus.cc:499] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:38.146613 3007 raft_consensus.cc:521] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.147451 3007 leader_election.cc:290] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:38.147994 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3686f13df75044e3817bed5606bcc618" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:38.148020 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3686f13df75044e3817bed5606bcc618" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:38.148218 1075 raft_consensus.cc:2490] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:38.148245 1211 raft_consensus.cc:2490] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 0.
I20261118 14:58:38.148692 875 leader_election.cc:304] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:38.148973 3007 raft_consensus.cc:2826] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:38.149080 3007 raft_consensus.cc:499] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:38.149168 3007 raft_consensus.cc:3096] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.151134 3007 raft_consensus.cc:521] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.151628 3007 leader_election.cc:290] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:38.152056 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3686f13df75044e3817bed5606bcc618" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:38.152061 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3686f13df75044e3817bed5606bcc618" candidate_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:38.152230 1075 raft_consensus.cc:3096] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.152248 1211 raft_consensus.cc:3096] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.154027 1075 raft_consensus.cc:2490] T 3686f13df75044e3817bed5606bcc618 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:38.154410 875 leader_election.cc:304] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:38.154650 3007 raft_consensus.cc:2826] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:38.154793 3007 raft_consensus.cc:705] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 LEADER]: Becoming Leader. State: Replica: 7c53fe91804746e8bfcf743e0ab8ba6f, State: Running, Role: LEADER
I20261118 14:58:38.155077 1211 raft_consensus.cc:2490] T 3686f13df75044e3817bed5606bcc618 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c53fe91804746e8bfcf743e0ab8ba6f in term 1.
I20261118 14:58:38.155097 3007 consensus_queue.cc:238] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.158007 3006 consensus_queue.cc:1049] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:38.159512 3007 consensus_queue.cc:1049] T 3686f13df75044e3817bed5606bcc618 P 7c53fe91804746e8bfcf743e0ab8ba6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 14:58:38.162209 1145 connection.cc:573] server connection from 127.0.0.1:38770 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:38.162573 1146 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:38.164192 1011 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:38.164510 873 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:38.171818 1265 connection.cc:573] client connection to 127.0.72.189:46707 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 14:58:38.172498 1265 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.189:46707)
W20261118 14:58:38.173132 1265 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.189:46707)
W20261118 14:58:38.173314 1265 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.189:46707)
W20261118 14:58:38.174074 1265 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.189:46707)
W20261118 14:58:38.176257 2887 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 14:58:38.165697 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.189:46707 (local address 127.0.72.129:52837)
1118 14:58:38.166824 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
1118 14:58:38.166839 (+ 15us) client_negotiation.cc:175] Beginning negotiation
1118 14:58:38.166933 (+ 94us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 14:58:38.176211 (+ 9278us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":1047,"threads_started":1}
W20261118 14:58:38.176749 2992 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 14:58:38.166041 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.189:46707 (local address 127.0.72.130:54103)
1118 14:58:38.166262 (+ 221us) negotiation.cc:107] Waiting for socket to connect
1118 14:58:38.166279 (+ 17us) client_negotiation.cc:175] Beginning negotiation
1118 14:58:38.166398 (+ 119us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 14:58:38.176704 (+ 10306us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":129}
W20261118 14:58:38.176309 2899 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 14:58:38.163326 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.189:46707 (local address 127.0.72.131:54429)
1118 14:58:38.163444 (+ 118us) negotiation.cc:107] Waiting for socket to connect
1118 14:58:38.163461 (+ 17us) client_negotiation.cc:175] Beginning negotiation
1118 14:58:38.163563 (+ 102us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 14:58:38.176264 (+ 12701us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":31}
W20261118 14:58:38.176342 2845 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 14:58:38.175716 (+ 0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.189:46707 (local address 127.0.0.1:38954)
1118 14:58:38.175931 (+ 215us) negotiation.cc:107] Waiting for socket to connect
1118 14:58:38.175950 (+ 19us) client_negotiation.cc:175] Beginning negotiation
1118 14:58:38.176070 (+ 120us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 14:58:38.176285 (+ 215us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":52}
I20261118 14:58:38.414384 2792 raft_consensus.cc:499] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:38.414563 2792 raft_consensus.cc:521] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:38.415248 2792 leader_election.cc:290] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:38.415707 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1e1c16e124e54a9e934a3af82ff8f5a0" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:38.415787 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1e1c16e124e54a9e934a3af82ff8f5a0" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7" is_pre_election: true
I20261118 14:58:38.415922 937 raft_consensus.cc:2490] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:38.415994 1211 raft_consensus.cc:2490] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 0.
I20261118 14:58:38.416348 1010 leader_election.cc:304] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f; no voters:
I20261118 14:58:38.416591 2792 raft_consensus.cc:2826] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:38.416698 2792 raft_consensus.cc:499] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:38.416770 2792 raft_consensus.cc:3096] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.418778 2792 raft_consensus.cc:521] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:38.419296 2792 leader_election.cc:290] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Requested vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), c01f1b88968f4905ba5a401a81ade9f7 (127.0.72.131:45343)
I20261118 14:58:38.419646 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1e1c16e124e54a9e934a3af82ff8f5a0" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:38.419759 1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1e1c16e124e54a9e934a3af82ff8f5a0" candidate_uuid: "245c4fe5e44448d98db1eed6329d150d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01f1b88968f4905ba5a401a81ade9f7"
I20261118 14:58:38.419842 937 raft_consensus.cc:3096] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.419938 1211 raft_consensus.cc:3096] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.422684 937 raft_consensus.cc:2490] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:38.422701 1211 raft_consensus.cc:2490] T 1e1c16e124e54a9e934a3af82ff8f5a0 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 245c4fe5e44448d98db1eed6329d150d in term 1.
I20261118 14:58:38.423066 1012 leader_election.cc:304] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:38.423360 2792 raft_consensus.cc:2826] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:38.423697 2792 raft_consensus.cc:705] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [term 1 LEADER]: Becoming Leader. State: Replica: 245c4fe5e44448d98db1eed6329d150d, State: Running, Role: LEADER
I20261118 14:58:38.423995 2792 consensus_queue.cc:238] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } }
I20261118 14:58:38.426787 3053 consensus_queue.cc:1049] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:38.427238 2792 consensus_queue.cc:1049] T 1e1c16e124e54a9e934a3af82ff8f5a0 P 245c4fe5e44448d98db1eed6329d150d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:38.454547 3058 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:38.454790 3058 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:38.455415 3058 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:38.457276 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:38.457472 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:38.457577 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:38.457828 3058 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:58:38.483632 3051 raft_consensus.cc:499] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 14:58:38.483793 3051 raft_consensus.cc:521] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.484328 3051 leader_election.cc:290] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:38.484740 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "055fee087c2d4bd0a15b1b70d131d78d" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d" is_pre_election: true
I20261118 14:58:38.484844 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "055fee087c2d4bd0a15b1b70d131d78d" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" is_pre_election: true
I20261118 14:58:38.485039 937 raft_consensus.cc:2490] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:38.485013 1075 raft_consensus.cc:2490] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 0.
I20261118 14:58:38.485425 1145 leader_election.cc:304] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c53fe91804746e8bfcf743e0ab8ba6f, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:38.485690 3051 raft_consensus.cc:2826] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 14:58:38.485806 3051 raft_consensus.cc:499] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 14:58:38.485878 3051 raft_consensus.cc:3096] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.487792 3051 raft_consensus.cc:521] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.488305 3051 leader_election.cc:290] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Requested vote from peers 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257), 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463)
I20261118 14:58:38.488801 1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "055fee087c2d4bd0a15b1b70d131d78d" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245c4fe5e44448d98db1eed6329d150d"
I20261118 14:58:38.488865 937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "055fee087c2d4bd0a15b1b70d131d78d" candidate_uuid: "c01f1b88968f4905ba5a401a81ade9f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f"
I20261118 14:58:38.488955 1075 raft_consensus.cc:3096] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.489055 937 raft_consensus.cc:3096] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f [term 0 FOLLOWER]: Advancing to term 1
I20261118 14:58:38.491044 1075 raft_consensus.cc:2490] T 055fee087c2d4bd0a15b1b70d131d78d P 245c4fe5e44448d98db1eed6329d150d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:38.491088 937 raft_consensus.cc:2490] T 055fee087c2d4bd0a15b1b70d131d78d P 7c53fe91804746e8bfcf743e0ab8ba6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c01f1b88968f4905ba5a401a81ade9f7 in term 1.
I20261118 14:58:38.491451 1148 leader_election.cc:304] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245c4fe5e44448d98db1eed6329d150d, c01f1b88968f4905ba5a401a81ade9f7; no voters:
I20261118 14:58:38.491729 3051 raft_consensus.cc:2826] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 14:58:38.491851 3051 raft_consensus.cc:705] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [term 1 LEADER]: Becoming Leader. State: Replica: c01f1b88968f4905ba5a401a81ade9f7, State: Running, Role: LEADER
I20261118 14:58:38.492138 3051 consensus_queue.cc:238] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } }
I20261118 14:58:38.494382 3052 consensus_queue.cc:1049] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:38.494573 3051 consensus_queue.cc:1049] T 055fee087c2d4bd0a15b1b70d131d78d P c01f1b88968f4905ba5a401a81ade9f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 14:58:38.761231 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:58:38.761427 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:38.982506 3065 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:38.982743 3065 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:38.983480 3065 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:38.985266 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:38.985445 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:38.985549 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:38.985751 3065 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:58:39.509698 290 master-stress-test.cc:579] Killing test thread 1/25
I20261118 14:58:39.736313 3081 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:39.736495 3081 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:39.737095 3081 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:39.737943 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20261118 14:58:39.739041 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:39.739199 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:39.739305 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:39.739527 3081 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:58:40.324579 3092 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:58:40.324743 3092 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:40.325351 3092 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:58:40.326128 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:58:40.326323 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:40.326463 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:58:40.326719 3092 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:41.013033 3102 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:41.013206 3102 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:41.013772 3102 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:41.016484 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:41.016681 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:41.016825 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:41.017072 3102 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:58:42.617467 3132 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:42.617660 3132 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:42.618291 3132 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:42.620098 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:42.620304 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:42.620427 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:42.620642 3132 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:58:43.522296 3138 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:58:43.522502 3138 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:43.523217 3138 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:58:43.523916 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:58:43.524111 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:43.524258 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:58:43.524544 3138 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:43.763036 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:58:43.763229 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:58:43.993227 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 6: this message will repeat every 5th retry.
W20261118 14:58:44.264909 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 6: this message will repeat every 5th retry.
I20261118 14:58:46.059098 3189 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:46.059289 3189 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:46.060025 3189 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:46.060861 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20261118 14:58:46.062145 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:58:46.062340 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:58:46.062513 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:46.062760 3189 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:58:48.764734 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:58:48.764925 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:58:48.775957 3220 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:58:48.776170 3220 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:48.777081 3220 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:58:48.778054 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:58:48.778275 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:48.778417 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:58:48.778664 3220 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:49.209573 3236 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:49.209784 3236 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:49.210769 3236 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:49.213021 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:49.213188 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:49.213322 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:49.213622 3236 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20261118 14:58:53.749778 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 11: this message will repeat every 5th retry.
I20261118 14:58:53.766398 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:58:53.766605 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:58:54.644660 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 11: this message will repeat every 5th retry.
I20261118 14:58:55.951216 3341 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:58:55.951428 3341 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:58:55.952186 3341 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:58:55.953190 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 14:58:55.954403 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:58:55.954576 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:58:55.954716 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:58:55.954957 3341 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:58:56.991075 3356 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:58:56.991298 3356 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:58:56.992012 3356 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:58:56.992916 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:58:56.993084 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:58:56.993196 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:58:56.993444 3356 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:58:58.768241 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:58:58.768448 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:59:03.770056 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:03.770254 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:59:03.952075 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 16: this message will repeat every 5th retry.
W20261118 14:59:04.737931 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 16: this message will repeat every 5th retry.
I20261118 14:59:05.410476 3475 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:59:05.410688 3475 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:59:05.411536 3475 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:59:05.412400 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:59:05.412578 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:59:05.412681 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:59:05.412914 3475 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:59:06.160655 3497 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:59:06.160887 3497 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:59:06.161648 3497 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:59:06.162739 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 14:59:06.164212 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:59:06.164400 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:59:06.164512 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:59:06.164752 3497 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:59:08.771761 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:08.771983 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:59:09.482896 1259 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (105 consecutive failures): Network error: Failed to ping master at 127.0.72.189:46707: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:59:09.596722 986 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (69 consecutive failures): Network error: Failed to ping master at 127.0.72.189:46707: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:59:09.602594 1124 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (69 consecutive failures): Network error: Failed to ping master at 127.0.72.189:46707: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:59:11.896603 3575 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:59:11.896822 3575 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:59:11.897461 3575 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:59:11.898427 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:59:11.898756 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:59:11.898860 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:59:11.899097 3575 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:59:13.773646 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:13.773860 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:59:13.781342 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 21: this message will repeat every 5th retry.
I20261118 14:59:14.533383 3621 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:59:14.533581 3621 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:59:14.534400 3621 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:59:14.535372 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 14:59:14.536417 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 14:59:14.536597 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 14:59:14.536734 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:59:14.536962 3621 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20261118 14:59:15.128181 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 21: this message will repeat every 5th retry.
I20261118 14:59:18.775496 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:18.775696 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:59:22.503151 3713 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:59:22.503371 3713 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:59:22.504156 3713 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:59:22.505059 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:59:22.505246 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:59:22.505366 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:59:22.505590 3713 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:59:23.777140 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:23.777338 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:59:24.286790 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 26: this message will repeat every 5th retry.
W20261118 14:59:25.279107 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 26: this message will repeat every 5th retry.
I20261118 14:59:28.615689 3819 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:59:28.615887 3819 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:59:28.616737 3819 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:59:28.631294 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 14:59:28.632565 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:59:28.632786 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:59:28.632899 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:59:28.633144 3819 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:59:28.778676 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:28.778913 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:59:33.780423 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:33.780629 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:59:34.168870 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 31: this message will repeat every 5th retry.
W20261118 14:59:34.582315 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 31: this message will repeat every 5th retry.
I20261118 14:59:35.102284 3890 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:59:35.102531 3890 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:59:35.103298 3890 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:59:35.104212 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:59:35.104388 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:59:35.104509 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:59:35.104735 3890 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:59:38.783138 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:38.783392 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:59:39.308313 3925 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:59:39.308557 3925 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:59:39.309778 3925 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:59:39.311029 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:59:39.311247 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:59:39.311374 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:59:39.311745 3925 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 14:59:43.431067 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 36: this message will repeat every 5th retry.
I20261118 14:59:43.784996 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:43.785197 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:59:44.696086 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 36: this message will repeat every 5th retry.
I20261118 14:59:46.936447 4017 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:59:46.936684 4017 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:59:46.938129 4017 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:59:46.939038 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 14:59:46.940287 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:59:46.940515 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:59:46.940629 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:59:46.940819 4017 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:59:47.876466 4026 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 14:59:47.876662 4026 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 14:59:47.877377 4026 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 14:59:47.880635 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 14:59:47.880961 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 14:59:47.881124 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 14:59:47.881385 4026 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 14:59:48.786526 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:48.786708 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 14:59:53.787936 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:53.788120 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 14:59:54.241690 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 41: this message will repeat every 5th retry.
W20261118 14:59:54.419116 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 41: this message will repeat every 5th retry.
I20261118 14:59:56.915844 4120 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 14:59:56.916039 4120 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 14:59:56.916817 4120 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 14:59:56.917526 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 14:59:56.917677 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 14:59:56.917819 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 14:59:56.918040 4120 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 14:59:58.790310 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 14:59:58.790634 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:00:03.792096 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:03.792294 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:00:03.928503 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 46: this message will repeat every 5th retry.
W20261118 15:00:04.295720 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 46: this message will repeat every 5th retry.
I20261118 15:00:06.998850 4256 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:00:06.999100 4256 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:00:07.000085 4256 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:00:07.001391 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 15:00:07.002851 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:00:07.003165 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:00:07.003298 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:00:07.003574 4256 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:00:08.793570 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:08.793788 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:00:09.695636 1258 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42447 (326 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42447: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:00:09.796005 1123 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42447 (230 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42447: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:00:09.828649 986 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (189 consecutive failures): Network error: Failed to ping master at 127.0.72.189:46707: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:00:12.888461 4333 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:00:12.888636 4333 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:00:12.889561 4333 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:00:12.890508 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:00:12.891002 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:00:12.891131 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:00:12.891386 4333 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:00:13.795569 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:13.795790 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:00:14.400458 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 51: this message will repeat every 5th retry.
W20261118 15:00:14.620925 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 51: this message will repeat every 5th retry.
I20261118 15:00:18.797649 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:18.797848 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:00:21.818620 4465 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:00:21.818799 4465 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:00:21.819481 4465 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:00:21.820433 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:00:21.820595 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:00:21.820708 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:00:21.821385 4465 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 15:00:23.698014 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 56: this message will repeat every 5th retry.
I20261118 15:00:23.799247 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:23.799427 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:00:24.533334 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 56: this message will repeat every 5th retry.
I20261118 15:00:26.180155 4534 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:00:26.180349 4534 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:00:26.181226 4534 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:00:26.183151 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:00:26.184206 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:00:26.184365 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:00:26.184479 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:00:26.184675 4534 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:00:28.800969 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:28.801201 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:00:30.389216 4598 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:00:30.389403 4598 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:00:30.390182 4598 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:00:30.390987 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:00:30.391134 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:00:30.391251 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:00:30.391490 4598 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:00:33.669463 2701 maintenance_manager.cc:419] P 9b90720bc0b34e4994abf4fc6f9e59e1: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033333
I20261118 15:00:33.738677 2700 maintenance_manager.cc:643] P 9b90720bc0b34e4994abf4fc6f9e59e1: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.068s user 0.030s sys 0.004s Metrics: {"bytes_written":26961,"cfile_init":1,"dirs.queue_time_us":406,"dirs.run_cpu_time_us":218,"dirs.run_wall_time_us":801,"drs_written":1,"lbm_read_time_us":65,"lbm_reads_lt_1ms":4,"lbm_write_time_us":697,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":66,"thread_start_us":512,"threads_started":2,"wal-append.queue_time_us":335}
I20261118 15:00:33.739984 2701 maintenance_manager.cc:419] P 9b90720bc0b34e4994abf4fc6f9e59e1: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4943 bytes on disk
I20261118 15:00:33.740687 2700 maintenance_manager.cc:643] P 9b90720bc0b34e4994abf4fc6f9e59e1: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4}
I20261118 15:00:33.802608 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:33.802807 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:00:33.955394 4660 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:00:33.955585 4660 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:00:33.956307 4660 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:00:33.957012 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:00:33.958375 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:00:33.958652 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:00:33.958765 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:00:33.959003 4660 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20261118 15:00:34.061770 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 61: this message will repeat every 5th retry.
W20261118 15:00:34.330742 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 61: this message will repeat every 5th retry.
I20261118 15:00:38.804234 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:38.804417 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:00:43.529043 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 66: this message will repeat every 5th retry.
I20261118 15:00:43.806093 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:43.806298 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:00:44.883711 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 66: this message will repeat every 5th retry.
I20261118 15:00:46.779011 4870 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:00:46.779285 4870 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:00:46.780182 4870 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:00:46.781116 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:00:46.782219 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:00:46.782398 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:00:46.782557 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:00:46.782799 4870 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:00:48.130806 4886 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:00:48.130997 4886 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:00:48.131821 4886 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:00:48.132715 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:00:48.132905 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:00:48.133006 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:00:48.133217 4886 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:00:48.807874 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:48.808072 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:00:52.570746 4982 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:00:52.570916 4982 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:00:52.571465 4982 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:00:52.572151 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:00:52.573298 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:00:52.573467 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:00:52.573567 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:00:52.573770 4982 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20261118 15:00:53.589088 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 71: this message will repeat every 5th retry.
I20261118 15:00:53.809417 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:53.809588 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:00:54.482028 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 71: this message will repeat every 5th retry.
I20261118 15:00:57.684432 5079 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:00:57.684631 5079 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:00:57.685444 5079 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:00:57.686452 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:00:57.688416 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:00:57.688601 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:00:57.688705 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:00:57.688936 5079 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:00:58.811199 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:00:58.811389 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:03.361917 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 76: this message will repeat every 5th retry.
I20261118 15:01:03.812791 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:03.812988 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:04.611244 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 76: this message will repeat every 5th retry.
I20261118 15:01:06.328997 5240 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:01:06.329243 5240 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:01:06.330097 5240 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:01:06.331216 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:01:06.332401 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:01:06.332573 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:01:06.332717 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:01:06.332981 5240 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:01:07.486148 5263 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:01:07.486371 5263 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:01:07.487351 5263 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:01:07.488410 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:01:07.488816 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:01:07.488916 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:01:07.489143 5263 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:01:08.814395 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:08.814595 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:09.919049 1258 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42447 (514 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42447: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:01:09.979352 1124 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (309 consecutive failures): Network error: Failed to ping master at 127.0.72.189:46707: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:01:10.008414 985 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42447 (349 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42447: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:01:13.656051 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 81: this message will repeat every 5th retry.
I20261118 15:01:13.816084 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:13.816274 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:14.574261 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 81: this message will repeat every 5th retry.
I20261118 15:01:18.817910 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:18.818157 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:01:19.617117 5447 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:01:19.617331 5447 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:01:19.618091 5447 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:01:19.619104 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:01:19.619270 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:01:19.619410 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:01:19.619663 5447 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:01:22.661381 5540 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:01:22.661612 5540 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:01:22.662564 5540 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:01:22.664144 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:01:22.665433 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:01:22.665611 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:01:22.665740 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:01:22.665979 5540 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20261118 15:01:23.758045 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 86: this message will repeat every 5th retry.
I20261118 15:01:23.819773 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:23.819969 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:24.972766 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 86: this message will repeat every 5th retry.
I20261118 15:01:26.250648 5597 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:01:26.250826 5597 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:01:26.251570 5597 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:01:26.252511 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:01:26.252640 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:01:26.252751 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:01:26.252964 5597 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:01:27.309468 5625 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:01:27.309677 5625 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:01:27.310423 5625 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:01:27.312542 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:01:27.312747 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:01:27.312868 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:01:27.313107 5625 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:01:28.821384 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:28.821614 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:33.785916 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 91: this message will repeat every 5th retry.
I20261118 15:01:33.823226 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:33.823441 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:35.210636 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 91: this message will repeat every 5th retry.
I20261118 15:01:38.826041 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:38.826399 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:01:40.703346 5871 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:01:40.703511 5871 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:01:40.704142 5871 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:01:40.705361 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:01:40.705910 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:01:40.706023 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:01:40.706259 5871 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:01:41.436124 5887 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:01:41.436348 5887 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:01:41.437165 5887 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:01:41.438097 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 15:01:41.439231 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:01:41.439365 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:01:41.439507 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:01:41.439741 5887 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20261118 15:01:43.493181 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 96: this message will repeat every 5th retry.
I20261118 15:01:43.827944 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:43.828130 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:44.856491 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 96: this message will repeat every 5th retry.
I20261118 15:01:48.829921 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:48.830129 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:01:53.531865 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 101: this message will repeat every 5th retry.
I20261118 15:01:53.831838 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:53.832080 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:01:54.698173 6102 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:01:54.698390 6102 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:01:54.699142 6102 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:01:54.700060 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:01:54.700191 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:01:54.700301 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:01:54.700513 6102 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 15:01:55.107954 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 101: this message will repeat every 5th retry.
I20261118 15:01:58.833724 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:01:58.833930 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:02:01.163851 6196 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:02:01.164048 6196 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:02:01.164848 6196 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:02:01.166992 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:02:01.168267 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:02:01.168411 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:02:01.168552 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:02:01.168798 6196 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:02:01.302927 6188 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:02:01.303109 6188 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:02:01.303656 6188 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:02:01.304569 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:02:01.304697 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:02:01.304804 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:02:01.304999 6188 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:02:03.835582 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:03.835789 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:02:04.169551 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 106: this message will repeat every 5th retry.
W20261118 15:02:04.354815 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 106: this message will repeat every 5th retry.
I20261118 15:02:08.837476 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:08.837709 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:02:09.128984 6294 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:02:09.129179 6294 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:02:09.129801 6294 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:02:09.130790 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:02:09.131243 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:02:09.131358 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:02:09.131546 6294 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 15:02:10.183091 985 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42447 (469 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42447: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:02:10.197856 1123 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42447 (470 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42447: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:02:10.376608 1259 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (657 consecutive failures): Network error: Failed to ping master at 127.0.72.189:46707: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:02:13.612805 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 111: this message will repeat every 5th retry.
I20261118 15:02:13.839289 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:13.839478 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:02:14.777544 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 111: this message will repeat every 5th retry.
I20261118 15:02:18.840953 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:18.841120 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:02:19.075748 6482 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:02:19.075979 6482 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:02:19.077313 6482 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:02:19.077869 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:02:19.079094 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:02:19.079234 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:02:19.079352 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:02:19.079584 6482 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:02:23.842736 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:23.842942 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:02:23.848914 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 116: this message will repeat every 5th retry.
W20261118 15:02:24.399497 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 116: this message will repeat every 5th retry.
I20261118 15:02:25.989879 6590 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:02:25.990067 6590 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:02:25.990978 6590 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:02:25.991837 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:02:25.991976 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:02:25.992096 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:02:25.992331 6590 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:02:28.844511 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:28.844704 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:02:31.754558 6697 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:02:31.754736 6697 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:02:31.755467 6697 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:02:31.756338 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:02:31.756494 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:02:31.756582 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:02:31.756934 6697 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 15:02:33.750705 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 121: this message will repeat every 5th retry.
I20261118 15:02:33.846182 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:33.846415 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:02:34.871263 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 121: this message will repeat every 5th retry.
I20261118 15:02:38.391675 6812 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:02:38.391891 6812 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:02:38.392766 6812 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:02:38.394227 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:02:38.395341 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:02:38.395510 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:02:38.395637 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:02:38.395890 6812 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:02:38.847676 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:38.847903 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:02:40.717181 6846 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:02:40.717381 6846 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:02:40.718071 6846 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:02:40.719040 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:02:40.719395 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:02:40.719534 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:02:40.719775 6846 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:02:43.849557 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:43.849756 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:02:43.851799 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 126: this message will repeat every 5th retry.
W20261118 15:02:44.464339 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 126: this message will repeat every 5th retry.
I20261118 15:02:44.802527 6919 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:02:44.802704 6919 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:02:44.803381 6919 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:02:44.804272 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:02:44.804419 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:02:44.804524 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:02:44.804726 6919 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:02:48.851199 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:48.851401 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:02:53.205967 7059 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:02:53.206179 7059 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:02:53.207142 7059 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:02:53.208082 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:02:53.209311 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:02:53.209523 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:02:53.209643 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:02:53.209903 7059 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:02:53.852973 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:53.853169 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:02:54.097869 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 131: this message will repeat every 5th retry.
W20261118 15:02:54.323057 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 131: this message will repeat every 5th retry.
I20261118 15:02:55.749012 7096 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:02:55.749166 7096 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:02:55.749759 7096 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:02:55.750662 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:02:55.750823 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:02:55.750912 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:02:55.751116 7096 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:02:58.854806 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:02:58.854998 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:02:59.864509 7164 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:02:59.864725 7164 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:02:59.865633 7164 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:02:59.866755 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:02:59.866894 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:02:59.867007 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:02:59.867319 7164 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:03:01.946727 7207 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:03:01.946920 7207 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:03:01.947574 7207 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:03:01.948521 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:03:01.949754 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:03:01.949898 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:03:01.950039 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:03:01.950300 7207 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:03:03.856678 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:03:03.856871 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:03:04.369665 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 136: this message will repeat every 5th retry.
W20261118 15:03:04.685781 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 136: this message will repeat every 5th retry.
I20261118 15:03:08.199791 7314 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:03:08.200006 7314 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:03:08.200740 7314 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:03:08.201768 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:03:08.203127 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:03:08.203299 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:03:08.203439 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:03:08.203672 7314 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:03:08.858458 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:03:08.858767 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:03:10.387872 985 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42447 (589 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42447: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:03:10.413002 1124 heartbeater.cc:648] Failed to heartbeat to 127.0.72.189:46707 (549 consecutive failures): Network error: Failed to ping master at 127.0.72.189:46707: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:03:10.468325 1258 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42447 (878 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42447: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:03:12.727337 7394 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:03:12.727541 7394 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:03:12.728142 7394 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:03:12.729055 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:03:12.729190 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:03:12.729310 1145 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:03:12.729516 7394 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:03:12.840777 7397 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:03:12.840966 7397 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:03:12.841687 7397 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:03:12.843982 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:03:12.844213 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:03:12.844337 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:03:12.844560 7397 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:03:13.860428 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:03:13.860643 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:03:14.400374 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 141: this message will repeat every 5th retry.
W20261118 15:03:14.567536 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 141: this message will repeat every 5th retry.
I20261118 15:03:18.864542 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:03:18.864825 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:03:23.866786 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:03:23.866997 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:03:24.509977 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 146: this message will repeat every 5th retry.
W20261118 15:03:24.633296 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 146: this message will repeat every 5th retry.
I20261118 15:03:25.197641 7612 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:03:25.197837 7612 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:03:25.198596 7612 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:03:25.199693 2696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 15:03:25.200778 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
W20261118 15:03:25.200953 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
I20261118 15:03:25.201082 2696 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:03:25.201321 7612 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:03:28.869163 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:03:28.869383 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 15:03:32.468402 7711 raft_consensus.cc:499] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7c53fe91804746e8bfcf743e0ab8ba6f)
I20261118 15:03:32.468564 7711 raft_consensus.cc:521] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c53fe91804746e8bfcf743e0ab8ba6f" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 41463 } } peers { permanent_uuid: "c01f1b88968f4905ba5a401a81ade9f7" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 45343 } } peers { permanent_uuid: "245c4fe5e44448d98db1eed6329d150d" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 38257 } }
I20261118 15:03:32.469314 7711 leader_election.cc:290] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463), 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257)
W20261118 15:03:32.470081 1145 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c53fe91804746e8bfcf743e0ab8ba6f (127.0.72.129:41463): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
W20261118 15:03:32.470216 1148 leader_election.cc:343] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Not found: Tablet not found: a78ecb3a620242f4a16128b0d456c352
I20261118 15:03:32.470319 1148 leader_election.cc:304] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c01f1b88968f4905ba5a401a81ade9f7; no voters: 245c4fe5e44448d98db1eed6329d150d, 7c53fe91804746e8bfcf743e0ab8ba6f
I20261118 15:03:32.470548 7711 raft_consensus.cc:2771] T a78ecb3a620242f4a16128b0d456c352 P c01f1b88968f4905ba5a401a81ade9f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 15:03:33.870823 2679 master_runner.cc:187] Error getting master registration for 127.0.72.190:42447: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111),
I20261118 15:03:33.871011 2679 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 15:03:34.656683 1148 consensus_peers.cc:597] T 23b3e64e87dd4302a68a8cff81a03da1 P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 23b3e64e87dd4302a68a8cff81a03da1. This is attempt 151: this message will repeat every 5th retry.
W20261118 15:03:35.776500 1148 consensus_peers.cc:597] T 1b5222f6dc7f4757adbf34c3f6a2729e P c01f1b88968f4905ba5a401a81ade9f7 -> Peer 245c4fe5e44448d98db1eed6329d150d (127.0.72.130:38257): Couldn't send request to peer 245c4fe5e44448d98db1eed6329d150d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b5222f6dc7f4757adbf34c3f6a2729e. This is attempt 151: this message will repeat every 5th retry.
I20261118 15:03:37.046419 7819 raft_consensus.cc:499] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7c9ef96ff074450a879e66787486386d)
I20261118 15:03:37.046691 7819 raft_consensus.cc:521] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed65a3719634e519d783540afd6189e" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42447 } } peers { permanent_uuid: "7c9ef96ff074450a879e66787486386d" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 46707 } } peers { permanent_uuid: "9b90720bc0b34e4994abf4fc6f9e59e1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 33967 } }
I20261118 15:03:37.047938 7819 leader_election.cc:290] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447), 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707)
W20261118 15:03:37.049165 2697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 15:03:37.051193 2696 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0ed65a3719634e519d783540afd6189e (127.0.72.190:42447): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42447: connect: Connection refused (error 111)
W20261118 15:03:37.051442 2697 leader_election.cc:336] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 7c9ef96ff074450a879e66787486386d (127.0.72.189:46707): Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: connect: Connection refused (error 111)
I20261118 15:03:37.051594 2697 leader_election.cc:304] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b90720bc0b34e4994abf4fc6f9e59e1; no voters: 0ed65a3719634e519d783540afd6189e, 7c9ef96ff074450a879e66787486386d
I20261118 15:03:37.051880 7819 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 9b90720bc0b34e4994abf4fc6f9e59e1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 15:03:38.023515 2701 maintenance_manager.cc:419] P 9b90720bc0b34e4994abf4fc6f9e59e1: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4943 bytes on disk
I20261118 15:03:38.028581 2700 maintenance_manager.cc:643] P 9b90720bc0b34e4994abf4fc6f9e59e1: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s user 0.005s sys 0.000s Metrics: {"dirs.queue_time_us":271,"thread_start_us":376,"threads_started":2,"wal-append.queue_time_us":284}
W20261118 15:03:38.378719 1266 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
W20261118 15:03:38.379004 1266 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
W20261118 15:03:38.379125 1266 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
W20261118 15:03:38.379269 1266 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
*** Aborted at 1795014218 (unix time) try "date -d @1795014218" if you are using GNU date ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389952 1295 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394560 1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000122) received by PID 290 (TID 0x7fc2c9858700) from PID 290; stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394560 1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395761 1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d0b14f4adfe3441c88447b2fc1e2d506 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394560 1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395761 1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d0b14f4adfe3441c88447b2fc1e2d506 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395804 1292 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394560 1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395761 1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d0b14f4adfe3441c88447b2fc1e2d506 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395804 1292 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.396973 1294 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
@ 0x7fc2e6d40980 (unknown) at ??:0
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394560 1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395761 1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d0b14f4adfe3441c88447b2fc1e2d506 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395804 1292 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.396973 1294 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397123 1274 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d24daba84f8c4c94824da319564f0453 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394560 1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395761 1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d0b14f4adfe3441c88447b2fc1e2d506 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395804 1292 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.396973 1294 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397123 1274 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d24daba84f8c4c94824da319564f0453 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397166 1281 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4dba69207cca45529e828162200c13f5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394560 1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395761 1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d0b14f4adfe3441c88447b2fc1e2d506 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395804 1292 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.396973 1294 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397123 1274 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d24daba84f8c4c94824da319564f0453 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397166 1281 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4dba69207cca45529e828162200c13f5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397289 1285 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
@ 0x7fc2dfab0e87 gsignal at ??:0
F20261118 15:03:38.389953 1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390090 1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390224 1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4789eba8c66842f6ae326364062589fb on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.390405 1290 master-stress-test.cc:299] Check failed: _s.ok() Bad status: F20261118 15:Tim38 out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104) .390427 1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_053874e1edfe4b8eac52b4ba5f1b1870 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391521 1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391639 1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_36b9017c9ae24ecdb8ae81bf85341052 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392022 1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0cbce4af337d4a5baeb313a32662d446 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.391993 1287 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392135 1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_5c2ab051f54546cab77c75d60b2a8bd1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392227 1291 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.392001 1276 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_7e3ec5e0d6054fb19bd740e064a6bc3f on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394488 1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.394560 1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395761 1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d0b14f4adfe3441c88447b2fc1e2d506 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.395804 1292 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.396973 1294 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397123 1274 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_d24daba84f8c4c94824da319564f0453 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397166 1281 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_4dba69207cca45529e828162200c13f5 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.397289 1285 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)F20261118 15:03:38.398447 1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_6c65bee0620c494591ca0d8d3226c6a2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42447,127.0.72.189:46707,127.0.72.188:33967, num_attempts: 771) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.0.72.189:46707: BlockingWrite error: write error: Connection reset by peer (error 104)
*** Check failure stack trace: ***
@ 0x7fc2dfab27f1 abort at ??:0
@ 0x7fc2e27ffd5d google::LogMessage::Fail() at ??:0
@ 0x7fc2e2803b23 google::LogMessage::SendToLog() at ??:0
@ 0x7fc2e27ff75c google::LogMessage::Flush() at ??:0
@ 0x7fc2e2800ee9 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x40b4d5 kudu::MasterStressTest::ReplaceTabletThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:393
@ 0x447484 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
@ 0x44733d std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
@ 0x4472fa _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
@ 0x4472ae std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
@ 0x446fe9 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
@ 0x7fc2e013a6df (unknown) at ??:0
@ 0x7fc2e6d356db start_thread at ??:0
@ 0x7fc2dfb9361f clone at ??:0