Diagnosed failure

CatalogManagerConfigurations/MasterStressTest.Test/2: Unrecognized error type. Please see the error log for more information.

Full log

Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN      ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-11-18T08:54:07Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T08:54:07Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task3eSafg/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task3eSafg/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:derby:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/metadb;create=true
Metastore Connection Driver :	 org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:	 APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 

 
 
 
 

 
 
 
 


 
 
 

 
 
 


 
 
 
 
 
 
 


 
 
 

 
 
 

 
 

 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 


 
 
 
 
 
 



 
 
 
 
 


 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 


Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/security.properties
2026-11-18 08:54:14: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task3eSafg/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task3eSafg/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20261118 08:54:21.594707   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--webserver_interface=127.0.72.190
--webserver_port=0
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 08:54:21.945640   572 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:21.945945   572 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:21.946048   572 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:21.954403   572 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:21.954492   572 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:21.954558   572 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:21.954618   572 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:21.965039   572 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:21.966674   572 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:21.968484   572 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:21.977042   577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:21.977046   578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:21.977902   580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:21.978412   572 server_base.cc:1061] running on GCE node
I20261118 08:54:21.979213   572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:21.980720   572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:21.981925   572 hybrid_clock.cc:648] HybridClock initialized: now 1794992061981836 us; error 80 us; skew 500 ppm
I20261118 08:54:21.982242   572 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:21.984352   572 webserver.cc:577] Webserver started at http://127.0.72.190:41231/ using document root <none> and password file <none>
I20261118 08:54:21.985061   572 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:21.985170   572 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:21.985992   572 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 08:54:21.988420   572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/instance:
uuid: "78d4026bc230421ea10b8c68a9ef7871"
format_stamp: "Formatted at 2026-11-18 08:54:21 on dist-test-slave-m8c8"
I20261118 08:54:21.989200   572 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal/instance:
uuid: "78d4026bc230421ea10b8c68a9ef7871"
format_stamp: "Formatted at 2026-11-18 08:54:21 on dist-test-slave-m8c8"
I20261118 08:54:21.994464   572 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.001s
I20261118 08:54:21.997740   586 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:21.999176   572 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20261118 08:54:21.999315   572 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
uuid: "78d4026bc230421ea10b8c68a9ef7871"
format_stamp: "Formatted at 2026-11-18 08:54:21 on dist-test-slave-m8c8"
I20261118 08:54:21.999569   572 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:22.022874   572 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:22.023790   572 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:22.024034   572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:22.042620   572 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42341
I20261118 08:54:22.042650   637 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42341 every 8 connection(s)
I20261118 08:54:22.044080   572 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
I20261118 08:54:22.048715   638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:22.051168   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 572
I20261118 08:54:22.051429   290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal/instance
I20261118 08:54:22.053730   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--webserver_interface=127.0.72.189
--webserver_port=0
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 08:54:22.054512   638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } has no permanent_uuid. Determining permanent_uuid...
I20261118 08:54:22.070093   638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } has no permanent_uuid. Determining permanent_uuid...
W20261118 08:54:22.071106   588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.189:33939: connect: Connection refused (error 111)
W20261118 08:54:22.072595   638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:33939: Network error: Client connection negotiation failed: client connection to 127.0.72.189:33939: connect: Connection refused (error 111)
I20261118 08:54:22.121907   638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } attempt: 1
W20261118 08:54:22.123939   638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:33939: Network error: Client connection negotiation failed: client connection to 127.0.72.189:33939: connect: Connection refused (error 111)
I20261118 08:54:22.192175   638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } attempt: 2
W20261118 08:54:22.193980   638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:33939: Network error: Client connection negotiation failed: client connection to 127.0.72.189:33939: connect: Connection refused (error 111)
I20261118 08:54:22.285224   638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } attempt: 3
W20261118 08:54:22.287227   638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:33939: Network error: Client connection negotiation failed: client connection to 127.0.72.189:33939: connect: Connection refused (error 111)
W20261118 08:54:22.416447   639 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:22.416770   639 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:22.416881   639 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:22.425249   639 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:22.425350   639 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:22.425828   639 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:22.425904   639 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:22.430450   638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } attempt: 4
W20261118 08:54:22.432528   638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.189:33939: Network error: Client connection negotiation failed: client connection to 127.0.72.189:33939: connect: Connection refused (error 111)
I20261118 08:54:22.436986   639 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:22.438602   639 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:22.440388   639 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:22.448717   650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:22.448680   647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:22.448748   648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:22.449851   639 server_base.cc:1061] running on GCE node
I20261118 08:54:22.450627   639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:22.452128   639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:22.453328   639 hybrid_clock.cc:648] HybridClock initialized: now 1794992062453275 us; error 41 us; skew 500 ppm
I20261118 08:54:22.453647   639 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:22.455760   639 webserver.cc:577] Webserver started at http://127.0.72.189:35929/ using document root <none> and password file <none>
I20261118 08:54:22.456449   639 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:22.456557   639 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:22.456921   639 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 08:54:22.459403   639 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/instance:
uuid: "857a577b88ba46cc846ecd1a25a722ca"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:22.460214   639 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal/instance:
uuid: "857a577b88ba46cc846ecd1a25a722ca"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:22.466146   639 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.003s
I20261118 08:54:22.469712   656 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:22.471447   639 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20261118 08:54:22.471628   639 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
uuid: "857a577b88ba46cc846ecd1a25a722ca"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:22.471835   639 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:22.489454   639 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:22.490382   639 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:22.490655   639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:22.511423   639 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:33939
I20261118 08:54:22.511436   707 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:33939 every 8 connection(s)
I20261118 08:54:22.512964   639 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
I20261118 08:54:22.517221   708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:22.520051   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 639
I20261118 08:54:22.520226   290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal/instance
I20261118 08:54:22.521242   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:41259
--webserver_interface=127.0.72.188
--webserver_port=0
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 08:54:22.523514   708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } has no permanent_uuid. Determining permanent_uuid...
I20261118 08:54:22.537590   708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } has no permanent_uuid. Determining permanent_uuid...
I20261118 08:54:22.546088   708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } has no permanent_uuid. Determining permanent_uuid...
W20261118 08:54:22.547586   657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:22.549081   708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:22.598363   708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } attempt: 1
W20261118 08:54:22.600262   708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:22.668488   708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } attempt: 2
W20261118 08:54:22.670584   708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:22.731792   638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } attempt: 5
I20261118 08:54:22.738198   638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } has no permanent_uuid. Determining permanent_uuid...
W20261118 08:54:22.740202   638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:22.761814   708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } attempt: 3
W20261118 08:54:22.763927   708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:22.791471   638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } attempt: 1
W20261118 08:54:22.793439   638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:22.861689   638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } attempt: 2
W20261118 08:54:22.863569   638 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:22.869073   709 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:22.869367   709 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:22.869447   709 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:22.878365   709 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:22.878460   709 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:22.878511   709 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:22.878556   709 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:22.889273   709 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.188:41259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.72.188
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:22.890918   709 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:22.892711   709 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:22.901058   718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:22.901014   717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:22.901723   720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:22.902417   709 server_base.cc:1061] running on GCE node
I20261118 08:54:22.903169   709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:22.904649   709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:22.905846   709 hybrid_clock.cc:648] HybridClock initialized: now 1794992062905775 us; error 64 us; skew 500 ppm
I20261118 08:54:22.906184   709 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:22.907152   708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } attempt: 4
I20261118 08:54:22.908560   709 webserver.cc:577] Webserver started at http://127.0.72.188:40343/ using document root <none> and password file <none>
W20261118 08:54:22.908911   708 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:22.909303   709 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:22.909402   709 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:22.909785   709 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 08:54:22.912179   709 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/data/instance:
uuid: "6c7d9f52503f41e4bafe2821e6fd6af1"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:22.912923   709 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/wal/instance:
uuid: "6c7d9f52503f41e4bafe2821e6fd6af1"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:22.918246   709 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20261118 08:54:22.921588   726 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:22.923233   709 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20261118 08:54:22.923455   709 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/wal
uuid: "6c7d9f52503f41e4bafe2821e6fd6af1"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:22.923692   709 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:22.938583   709 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:22.939836   709 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:22.940186   709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:22.962540   709 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.188:41259
I20261118 08:54:22.962553   777 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.188:41259 every 8 connection(s)
I20261118 08:54:22.964308   709 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/data/info.pb
I20261118 08:54:22.966774   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 709
I20261118 08:54:22.966959   290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-2/wal/instance
I20261118 08:54:22.969305   778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:22.969786   638 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } attempt: 3
I20261118 08:54:22.979097   778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } has no permanent_uuid. Determining permanent_uuid...
I20261118 08:54:22.999146   638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap starting.
I20261118 08:54:22.999230   778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } has no permanent_uuid. Determining permanent_uuid...
I20261118 08:54:23.003113   638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:23.004325   638 log.cc:826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:23.007369   778 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } has no permanent_uuid. Determining permanent_uuid...
I20261118 08:54:23.008410   638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: No bootstrap required, opened a new log
I20261118 08:54:23.015676   638 raft_consensus.cc:360] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.015950   638 raft_consensus.cc:386] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:23.016064   638 raft_consensus.cc:748] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Initialized, Role: FOLLOWER
I20261118 08:54:23.016714   638 consensus_queue.cc:261] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.017597   786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.017895   786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:23.018667   778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: Bootstrap starting.
I20261118 08:54:23.018841   638 sys_catalog.cc:565] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:23.022773   778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:23.024398   778 log.cc:826] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: Log is configured to *not* fsync() on all Append() calls
W20261118 08:54:23.027036   592 tablet.cc:2441] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 08:54:23.027209   778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: No bootstrap required, opened a new log
I20261118 08:54:23.035250   778 raft_consensus.cc:360] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.035738   778 raft_consensus.cc:386] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:23.035916   778 raft_consensus.cc:748] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7d9f52503f41e4bafe2821e6fd6af1, State: Initialized, Role: FOLLOWER
I20261118 08:54:23.036852   778 consensus_queue.cc:261] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.043466   572 master_runner.cc:187] Error getting master registration for 127.0.72.189:33939: OK, instance_id { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" instance_seqno: 1794992062505753 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20261118 08:54:23.050841   803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.051213   803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:23.052399   778 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:23.054576   572 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:54:23.054319   801 catalog_manager.cc:1609] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20261118 08:54:23.054755   801 catalog_manager.cc:898] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20261118 08:54:23.078428   786 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:23.078748   786 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.080739   786 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:23.082221   753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" is_pre_election: true
I20261118 08:54:23.082841   753 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 0.
I20261118 08:54:23.089746   587 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871; no voters: 
I20261118 08:54:23.090960   786 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:23.091135   786 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:23.091241   786 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:23.094861   786 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.097041   753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1"
I20261118 08:54:23.097283   753 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:23.100651   753 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 1.
I20261118 08:54:23.101648   587 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871; no voters: 
I20261118 08:54:23.109010   786 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 1 election: Requested vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:23.110280   786 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:23.110991   786 raft_consensus.cc:705] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 1 LEADER]: Becoming Leader. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Running, Role: LEADER
I20261118 08:54:23.113349   786 consensus_queue.cc:238] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.119354   813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 1 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.119717   813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:23.132269   821 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20261118 08:54:23.134711   819 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 08:54:23.137976   819 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 08:54:23.167618   803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 1 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.167964   803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:23.182780   786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.183100   786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:23.195333   803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.195598   803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:23.197758   709 master_runner.cc:187] Error getting master registration for 127.0.72.189:33939: OK, instance_id { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" instance_seqno: 1794992062505753 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20261118 08:54:23.208113   708 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } attempt: 5
I20261118 08:54:23.211732   709 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:23.221289   708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap starting.
I20261118 08:54:23.225664   708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:23.227665   708 log.cc:826] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Log is configured to *not* fsync() on all Append() calls
W20261118 08:54:23.231904   826 catalog_manager.cc:1609] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20261118 08:54:23.233011   826 catalog_manager.cc:898] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20261118 08:54:23.237808   708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: No bootstrap required, opened a new log
I20261118 08:54:23.244191   819 catalog_manager.cc:1398] Generated new cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:23.244314   819 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 08:54:23.246789   708 raft_consensus.cc:360] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.247244   708 raft_consensus.cc:386] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:23.247762   708 raft_consensus.cc:748] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857a577b88ba46cc846ecd1a25a722ca, State: Initialized, Role: FOLLOWER
I20261118 08:54:23.248679   708 consensus_queue.cc:261] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.255983   833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.258185   683 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca" is_pre_election: true
I20261118 08:54:23.258659   681 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:23.258157   708 sys_catalog.cc:565] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:23.261642   833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:23.263510   821 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:23.263507   682 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca"
I20261118 08:54:23.267019   682 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 1.
I20261118 08:54:23.285120   833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 1 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.285465   833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:23.286087   837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.286484   837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:23.289351   786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:23.289580   786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:23.294399   819 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 08:54:23.296689   819 catalog_manager.cc:1555] Loading token signing keys...
I20261118 08:54:23.323945   819 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Generated new TSK 0
I20261118 08:54:23.325229   819 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 08:54:23.326709   819 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 08:54:23.331487   819 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 08:54:23.356958   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.72.129:0
--local_ip_for_outbound_sockets=127.0.72.129
--webserver_interface=127.0.72.129
--webserver_port=0
--tserver_master_addrs=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 08:54:23.394093   853 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:23.394212   853 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: loading cluster ID for follower catalog manager: success
I20261118 08:54:23.398656   853 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: acquiring CA information for follower catalog manager: success
I20261118 08:54:23.413873   853 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:23.845132   856 raft_consensus.cc:499] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:23.845541   856 raft_consensus.cc:521] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:23.847283   856 leader_election.cc:290] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939)
I20261118 08:54:23.847967   681 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca" is_pre_election: true
I20261118 08:54:23.847961   613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "78d4026bc230421ea10b8c68a9ef7871" is_pre_election: true
I20261118 08:54:23.848855   728 leader_election.cc:304] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7d9f52503f41e4bafe2821e6fd6af1; no voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca
I20261118 08:54:23.849653   856 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:54:23.925673   855 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:23.926107   855 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:23.926268   855 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:23.939601   855 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:23.939872   855 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.72.129
I20261118 08:54:23.956419   855 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.72.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.72.129
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.72.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:23.958516   855 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:23.961050   855 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:23.972324   861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:23.975593   862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:23.983698   864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:23.983942   855 server_base.cc:1061] running on GCE node
I20261118 08:54:23.984627   855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:23.985702   855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:23.986915   855 hybrid_clock.cc:648] HybridClock initialized: now 1794992063986857 us; error 50 us; skew 500 ppm
I20261118 08:54:23.987224   855 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:23.990185   855 webserver.cc:577] Webserver started at http://127.0.72.129:36353/ using document root <none> and password file <none>
I20261118 08:54:23.990918   855 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:23.991019   855 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:23.991415   855 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 08:54:23.993945   855 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/data/instance:
uuid: "ad2789ff5b54489c84966cff78be4721"
format_stamp: "Formatted at 2026-11-18 08:54:23 on dist-test-slave-m8c8"
I20261118 08:54:23.994714   855 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/instance:
uuid: "ad2789ff5b54489c84966cff78be4721"
format_stamp: "Formatted at 2026-11-18 08:54:23 on dist-test-slave-m8c8"
I20261118 08:54:24.000763   855 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.004s
I20261118 08:54:24.004264   870 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:24.005724   855 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20261118 08:54:24.005887   855 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal
uuid: "ad2789ff5b54489c84966cff78be4721"
format_stamp: "Formatted at 2026-11-18 08:54:23 on dist-test-slave-m8c8"
I20261118 08:54:24.006083   855 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:24.017926   855 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:24.018719   855 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:24.018945   855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:24.019871   855 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 08:54:24.021988   855 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 08:54:24.022079   855 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:24.022181   855 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 08:54:24.022250   855 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:24.104028   855 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.129:39287
I20261118 08:54:24.104095   982 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.129:39287 every 8 connection(s)
I20261118 08:54:24.105548   855 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/data/info.pb
I20261118 08:54:24.115700   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 855
I20261118 08:54:24.115948   290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/instance
I20261118 08:54:24.123787   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.72.130:0
--local_ip_for_outbound_sockets=127.0.72.130
--webserver_interface=127.0.72.130
--webserver_port=0
--tserver_master_addrs=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 08:54:24.124344   983 heartbeater.cc:345] Connected to a master server at 127.0.72.188:41259
I20261118 08:54:24.124522   985 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:24.124763   983 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:24.124763   985 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:24.125890   983 heartbeater.cc:508] Master 127.0.72.188:41259 requested a full tablet report, sending...
I20261118 08:54:24.125861   985 heartbeater.cc:508] Master 127.0.72.189:33939 requested a full tablet report, sending...
I20261118 08:54:24.125838   984 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:24.126375   984 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:24.126953   984 heartbeater.cc:508] Master 127.0.72.190:42341 requested a full tablet report, sending...
I20261118 08:54:24.128551   743 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:24.128639   673 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:24.128683   603 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:24.130394   603 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.0.72.129:51453
I20261118 08:54:24.234964   826 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:24.235057   826 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: loading cluster ID for follower catalog manager: success
I20261118 08:54:24.237641   826 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: acquiring CA information for follower catalog manager: success
I20261118 08:54:24.239377   826 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:24.248214   994 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:24.248431   994 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:24.249696   994 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:24.250236   613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857a577b88ba46cc846ecd1a25a722ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "78d4026bc230421ea10b8c68a9ef7871" is_pre_election: true
I20261118 08:54:24.250316   753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857a577b88ba46cc846ecd1a25a722ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" is_pre_election: true
I20261118 08:54:24.250979   657 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:24.251348   994 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:54:24.470762   993 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:24.471072   993 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:24.471210   993 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:24.480082   993 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:24.480270   993 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.72.130
I20261118 08:54:24.490489   993 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.72.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.72.130
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.72.130

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:24.492208   993 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:24.494048   993 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:24.504308  1000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:24.504444   999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:24.504972  1002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:24.505577   993 server_base.cc:1061] running on GCE node
I20261118 08:54:24.506135   993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:24.507134   993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:24.508333   993 hybrid_clock.cc:648] HybridClock initialized: now 1794992064508259 us; error 68 us; skew 500 ppm
I20261118 08:54:24.508625   993 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:24.510792   993 webserver.cc:577] Webserver started at http://127.0.72.130:45439/ using document root <none> and password file <none>
I20261118 08:54:24.511857   993 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:24.511994   993 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:24.512414   993 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 08:54:24.515533   993 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/data/instance:
uuid: "213b5f7a64e74b109a3a9387e5975042"
format_stamp: "Formatted at 2026-11-18 08:54:24 on dist-test-slave-m8c8"
I20261118 08:54:24.516551   993 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/instance:
uuid: "213b5f7a64e74b109a3a9387e5975042"
format_stamp: "Formatted at 2026-11-18 08:54:24 on dist-test-slave-m8c8"
I20261118 08:54:24.523669   993 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.000s
I20261118 08:54:24.527768  1008 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:24.529428   993 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20261118 08:54:24.529603   993 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal
uuid: "213b5f7a64e74b109a3a9387e5975042"
format_stamp: "Formatted at 2026-11-18 08:54:24 on dist-test-slave-m8c8"
I20261118 08:54:24.529810   993 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:24.552639   993 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:24.553980   993 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:24.554227   993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:24.555228   993 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 08:54:24.557058   993 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 08:54:24.557161   993 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:24.557284   993 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 08:54:24.557360   993 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:24.590534   993 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.130:33649
I20261118 08:54:24.590580  1120 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.130:33649 every 8 connection(s)
I20261118 08:54:24.592084   993 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/data/info.pb
I20261118 08:54:24.593792   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 993
I20261118 08:54:24.593997   290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/instance
I20261118 08:54:24.598877   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.72.131:0
--local_ip_for_outbound_sockets=127.0.72.131
--webserver_interface=127.0.72.131
--webserver_port=0
--tserver_master_addrs=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 08:54:24.611102  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:24.611249  1123 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:24.611558  1122 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:24.611567  1123 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:24.612588  1122 heartbeater.cc:508] Master 127.0.72.190:42341 requested a full tablet report, sending...
I20261118 08:54:24.612615  1123 heartbeater.cc:508] Master 127.0.72.189:33939 requested a full tablet report, sending...
I20261118 08:54:24.613386  1121 heartbeater.cc:345] Connected to a master server at 127.0.72.188:41259
I20261118 08:54:24.613510  1121 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:24.614003  1121 heartbeater.cc:508] Master 127.0.72.188:41259 requested a full tablet report, sending...
I20261118 08:54:24.614339   603 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:24.614418   673 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:24.615123   743 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:24.615378   603 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.0.72.130:37065
I20261118 08:54:24.633255   984 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
I20261118 08:54:24.799680  1129 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:24.799882  1129 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:24.800585  1129 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:24.801195   753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857a577b88ba46cc846ecd1a25a722ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" is_pre_election: true
I20261118 08:54:24.801245   613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857a577b88ba46cc846ecd1a25a722ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "78d4026bc230421ea10b8c68a9ef7871" is_pre_election: true
I20261118 08:54:24.801867   658 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:24.802169  1129 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:54:24.964583  1128 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:24.964875  1128 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:24.964996  1128 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:24.973536  1128 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:24.973742  1128 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.72.131
I20261118 08:54:24.984437  1128 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.72.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.72.131
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.72.131

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:24.986058  1128 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:24.988209  1128 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:24.998422  1135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:24.998409  1134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:24.998462  1137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:24.999557  1128 server_base.cc:1061] running on GCE node
I20261118 08:54:25.000156  1128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:25.001127  1128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:25.002350  1128 hybrid_clock.cc:648] HybridClock initialized: now 1794992065002297 us; error 47 us; skew 500 ppm
I20261118 08:54:25.002650  1128 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:25.004901  1128 webserver.cc:577] Webserver started at http://127.0.72.131:40181/ using document root <none> and password file <none>
I20261118 08:54:25.005620  1128 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:25.005719  1128 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:25.006062  1128 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 08:54:25.008559  1128 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/data/instance:
uuid: "3934fbc351ee4a99add35742e90dfa68"
format_stamp: "Formatted at 2026-11-18 08:54:25 on dist-test-slave-m8c8"
I20261118 08:54:25.009317  1128 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/instance:
uuid: "3934fbc351ee4a99add35742e90dfa68"
format_stamp: "Formatted at 2026-11-18 08:54:25 on dist-test-slave-m8c8"
I20261118 08:54:25.014825  1128 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20261118 08:54:25.018841  1143 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:25.020363  1128 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20261118 08:54:25.020534  1128 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal
uuid: "3934fbc351ee4a99add35742e90dfa68"
format_stamp: "Formatted at 2026-11-18 08:54:25 on dist-test-slave-m8c8"
I20261118 08:54:25.020728  1128 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:25.038028  1128 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:25.038931  1128 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:25.039170  1128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:25.040112  1128 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 08:54:25.041785  1128 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 08:54:25.041877  1128 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:25.041978  1128 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 08:54:25.042048  1128 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:25.079561  1128 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.131:46447
I20261118 08:54:25.079612  1255 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.131:46447 every 8 connection(s)
I20261118 08:54:25.080983  1128 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/data/info.pb
I20261118 08:54:25.089530   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 1128
I20261118 08:54:25.089743   290 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/instance
I20261118 08:54:25.107687  1256 heartbeater.cc:345] Connected to a master server at 127.0.72.188:41259
I20261118 08:54:25.107784  1258 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:25.107931  1257 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:25.108139  1257 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:25.108134  1256 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:25.108134  1258 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:25.109475  1256 heartbeater.cc:508] Master 127.0.72.188:41259 requested a full tablet report, sending...
I20261118 08:54:25.109488  1257 heartbeater.cc:508] Master 127.0.72.190:42341 requested a full tablet report, sending...
I20261118 08:54:25.109499  1258 heartbeater.cc:508] Master 127.0.72.189:33939 requested a full tablet report, sending...
I20261118 08:54:25.111363   743 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:25.111655   673 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:25.111681   603 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:25.112522   603 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.0.72.131:45591
I20261118 08:54:25.118196  1122 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
I20261118 08:54:25.119500   290 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20261118 08:54:25.152168   602 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_8b08d3e4ed824f1680377a6d5e7518a8"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.152191   601 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_1ee6cacf31104df284b253928039759b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.154697   600 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_567d34233a4b440a952301f19a6bcb10"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.155061   597 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_d2d73a819dc64fe0badbe6d84293bfff"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.155674   599 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_41d62c0cf6b54938b8e1770b859e56e8"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.159219   598 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_294f03c6875c4da0845888273ea80a2a"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.154533   603 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.159596   596 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_5f1fb0758cf1413eaed1d8b4717aa902"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.160017   595 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_ef26d2c593994e118ff3d01a0f5a9a1d"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.160173   594 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38576:
name: "default.Table_bc1f66261ac64a87912e19a32ce0ca44"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20261118 08:54:25.163425   601 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1ee6cacf31104df284b253928039759b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.163425   602 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8b08d3e4ed824f1680377a6d5e7518a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 08:54:25.199602   290 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task3eSafg/build/asan/bin/kudu pid 572
I20261118 08:54:25.201236   633 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 08:54:25.353176  1304 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:25.353583  1304 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:25.355222  1304 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:25.355268   753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857a577b88ba46cc846ecd1a25a722ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" is_pre_election: true
I20261118 08:54:25.361577   613 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857a577b88ba46cc846ecd1a25a722ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "78d4026bc230421ea10b8c68a9ef7871" is_pre_election: true
I20261118 08:54:25.362315   658 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:25.362802   290 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task3eSafg/build/asan/bin/kudu with pid 572
I20261118 08:54:25.367575  1304 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:54:25.381270  1264 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.190:42341)
I20261118 08:54:25.614893  1257 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
W20261118 08:54:25.616374  1257 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42341 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:25.621366  1122 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42341 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:25.637907   984 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42341 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:25.662045  1308 raft_consensus.cc:499] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:25.662232  1308 raft_consensus.cc:521] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:25.662938  1308 leader_election.cc:290] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939)
W20261118 08:54:25.663913   728 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:25.664062   681 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca" is_pre_election: true
I20261118 08:54:25.664297   681 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7d9f52503f41e4bafe2821e6fd6af1 in term 1.
I20261118 08:54:25.664844   728 leader_election.cc:304] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 857a577b88ba46cc846ecd1a25a722ca; no voters: 
I20261118 08:54:25.665104  1308 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261118 08:54:25.665249  1308 raft_consensus.cc:499] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:25.665371  1308 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 1 FOLLOWER]: Advancing to term 2
W20261118 08:54:25.665561   728 leader_election.cc:336] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:25.685961  1308 raft_consensus.cc:521] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:25.686921  1308 leader_election.cc:290] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [CANDIDATE]: Term 2 election: Requested vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939)
I20261118 08:54:25.688292   681 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca"
I20261118 08:54:25.688503   681 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 1 FOLLOWER]: Advancing to term 2
W20261118 08:54:25.689944   728 leader_election.cc:336] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:25.782953   681 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7d9f52503f41e4bafe2821e6fd6af1 in term 2.
I20261118 08:54:25.783748   728 leader_election.cc:304] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 857a577b88ba46cc846ecd1a25a722ca; no voters: 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:25.784811  1308 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 2 FOLLOWER]: Leader election won for term 2
I20261118 08:54:25.785426  1308 raft_consensus.cc:705] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7d9f52503f41e4bafe2821e6fd6af1, State: Running, Role: LEADER
I20261118 08:54:25.786054  1308 consensus_queue.cc:238] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:25.790704  1311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c7d9f52503f41e4bafe2821e6fd6af1. Latest consensus state: current_term: 2 leader_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:25.791229  1311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:25.792609  1315 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 08:54:25.792673   681 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 2 FOLLOWER]: Refusing update from remote peer 6c7d9f52503f41e4bafe2821e6fd6af1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20261118 08:54:25.793606   728 consensus_peers.cc:597] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 -> Peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Couldn't send request to peer 78d4026bc230421ea10b8c68a9ef7871. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:25.793665  1311 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20261118 08:54:25.795461  1304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c7d9f52503f41e4bafe2821e6fd6af1. Latest consensus state: current_term: 2 leader_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:25.795629  1304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:25.796083  1315 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 08:54:25.797735  1315 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:25.798377  1315 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 08:54:25.798554  1304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:25.798746  1304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:25.800001  1315 catalog_manager.cc:1555] Loading token signing keys...
I20261118 08:54:25.800213  1314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:25.800522  1314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:25.801040  1315 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: Loaded TSK: 0
I20261118 08:54:25.801913  1315 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 08:54:25.802723  1315 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 08:54:25.803512  1315 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 08:54:25.813833   743 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_bc1f66261ac64a87912e19a32ce0ca44"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.815349   742 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_ef26d2c593994e118ff3d01a0f5a9a1d"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.815629   741 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_5f1fb0758cf1413eaed1d8b4717aa902"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.815860   737 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_1ee6cacf31104df284b253928039759b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.816152   736 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_567d34233a4b440a952301f19a6bcb10"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.816012   738 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_8b08d3e4ed824f1680377a6d5e7518a8"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.815757   740 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_d2d73a819dc64fe0badbe6d84293bfff"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.816110   735 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_41d62c0cf6b54938b8e1770b859e56e8"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.816310   739 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:25.817056   734 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_294f03c6875c4da0845888273ea80a2a"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20261118 08:54:25.822751   743 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_bc1f66261ac64a87912e19a32ce0ca44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.826678   735 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_41d62c0cf6b54938b8e1770b859e56e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.827874   741 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_5f1fb0758cf1413eaed1d8b4717aa902 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.829022   737 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1ee6cacf31104df284b253928039759b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.830036   742 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_ef26d2c593994e118ff3d01a0f5a9a1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.831033   739 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c3f2df1b938946c19c1ca00d8fcbd640 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.832034   734 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_294f03c6875c4da0845888273ea80a2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.832958   738 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8b08d3e4ed824f1680377a6d5e7518a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.833933   740 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d2d73a819dc64fe0badbe6d84293bfff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 08:54:25.834869   736 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_567d34233a4b440a952301f19a6bcb10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 08:54:26.268656   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--webserver_interface=127.0.72.190
--webserver_port=41231
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 08:54:26.279533   743 catalog_manager.cc:2215] created HMS catalog entry for table  [id=4fcf984156584e30aac26283f71adda2]
I20261118 08:54:26.381810   735 catalog_manager.cc:2215] created HMS catalog entry for table  [id=a57cd9d759e94822ae1e9ecb7faad9f7]
I20261118 08:54:26.410624   743 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48432:
table { table_name: "default.Table_bc1f66261ac64a87912e19a32ce0ca44" } new_table_name: "default.Table_7fe6a97ebd294ce68601eaf9f252d0ee" modify_external_catalogs: true
I20261118 08:54:26.422962  1056 tablet_service.cc:1500] Processing CreateTablet for tablet 043a26a7d5004cc883eab2489a3ac5ba (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.424873  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043a26a7d5004cc883eab2489a3ac5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.427285   918 tablet_service.cc:1500] Processing CreateTablet for tablet 043a26a7d5004cc883eab2489a3ac5ba (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.429122   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043a26a7d5004cc883eab2489a3ac5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.447082   735 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48432:
table { table_name: "default.Table_41d62c0cf6b54938b8e1770b859e56e8" } new_table_name: "default.Table_eefa15fa899c4b1e9162efce0db7e4af" modify_external_catalogs: true
I20261118 08:54:26.450837  1191 tablet_service.cc:1500] Processing CreateTablet for tablet 043a26a7d5004cc883eab2489a3ac5ba (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.454574  1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 043a26a7d5004cc883eab2489a3ac5ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.455242  1055 tablet_service.cc:1500] Processing CreateTablet for tablet 8b0f9c9d6eef4b66aaab3bcafd810360 (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.455262  1190 tablet_service.cc:1500] Processing CreateTablet for tablet 8b0f9c9d6eef4b66aaab3bcafd810360 (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.455796  1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0f9c9d6eef4b66aaab3bcafd810360. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.455821  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0f9c9d6eef4b66aaab3bcafd810360. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.462121  1337 tablet_bootstrap.cc:492] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:26.467058  1337 tablet_bootstrap.cc:654] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.472889   917 tablet_service.cc:1500] Processing CreateTablet for tablet 8b0f9c9d6eef4b66aaab3bcafd810360 (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.473467  1336 tablet_bootstrap.cc:492] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:26.473568   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0f9c9d6eef4b66aaab3bcafd810360. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.478444  1336 tablet_bootstrap.cc:654] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.480154  1336 log.cc:826] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:26.480752  1337 log.cc:826] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:26.483359  1338 tablet_bootstrap.cc:492] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:26.489260   741 catalog_manager.cc:2215] created HMS catalog entry for table  [id=c2ea63be7ece4af4bcfb932bd174e957]
I20261118 08:54:26.495781  1338 tablet_bootstrap.cc:654] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.497337   741 master_service.cc:965] ReplaceTablet: received request to replace tablet 043a26a7d5004cc883eab2489a3ac5ba from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:26.497699  1338 log.cc:826] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:26.502333  1337 tablet_bootstrap.cc:492] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:26.502754  1337 ts_tablet_manager.cc:1449] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.000s
I20261118 08:54:26.516451   918 tablet_service.cc:1500] Processing CreateTablet for tablet 274fd2bd8f844da4904afc02a1b15b95 (DEFAULT_TABLE table=default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.516933   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274fd2bd8f844da4904afc02a1b15b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.517217  1055 tablet_service.cc:1500] Processing CreateTablet for tablet 274fd2bd8f844da4904afc02a1b15b95 (DEFAULT_TABLE table=default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.517638  1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274fd2bd8f844da4904afc02a1b15b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.518153  1337 raft_consensus.cc:360] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.518580  1337 raft_consensus.cc:386] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.518685  1337 raft_consensus.cc:748] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.522723  1190 tablet_service.cc:1500] Processing CreateTablet for tablet 274fd2bd8f844da4904afc02a1b15b95 (DEFAULT_TABLE table=default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.523186  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 274fd2bd8f844da4904afc02a1b15b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.528038   741 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: Sending DeleteTablet for 3 replicas of tablet 043a26a7d5004cc883eab2489a3ac5ba
I20261118 08:54:26.529388  1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 043a26a7d5004cc883eab2489a3ac5ba with delete_type TABLET_DATA_DELETED (replaced by tablet 641a0b1632204ce89ed10fa1ad6f8408) from {username='slave'} at 127.0.0.1:52346
I20261118 08:54:26.529937   741 catalog_manager.cc:6936] ReplaceTablet: tablet 043a26a7d5004cc883eab2489a3ac5ba deleted and replaced by tablet 641a0b1632204ce89ed10fa1ad6f8408
I20261118 08:54:26.530058   917 tablet_service.cc:1547] Processing DeleteTablet for tablet 043a26a7d5004cc883eab2489a3ac5ba with delete_type TABLET_DATA_DELETED (replaced by tablet 641a0b1632204ce89ed10fa1ad6f8408) from {username='slave'} at 127.0.0.1:44724
I20261118 08:54:26.530090  1191 tablet_service.cc:1547] Processing DeleteTablet for tablet 043a26a7d5004cc883eab2489a3ac5ba with delete_type TABLET_DATA_DELETED (replaced by tablet 641a0b1632204ce89ed10fa1ad6f8408) from {username='slave'} at 127.0.0.1:59210
I20261118 08:54:26.535815  1337 consensus_queue.cc:261] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.540349   741 master_service.cc:965] ReplaceTablet: received request to replace tablet 043a26a7d5004cc883eab2489a3ac5ba from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:26.542833   918 tablet_service.cc:1500] Processing CreateTablet for tablet 641a0b1632204ce89ed10fa1ad6f8408 (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.542965   741 master_service.cc:965] ReplaceTablet: received request to replace tablet 043a26a7d5004cc883eab2489a3ac5ba from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:26.543412   741 master_service.cc:965] ReplaceTablet: received request to replace tablet 043a26a7d5004cc883eab2489a3ac5ba from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:26.543782   741 master_service.cc:965] ReplaceTablet: received request to replace tablet 043a26a7d5004cc883eab2489a3ac5ba from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:26.544134   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 641a0b1632204ce89ed10fa1ad6f8408. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.546213  1054 tablet_service.cc:1500] Processing CreateTablet for tablet 641a0b1632204ce89ed10fa1ad6f8408 (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.546729  1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 641a0b1632204ce89ed10fa1ad6f8408. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.548483  1190 tablet_service.cc:1500] Processing CreateTablet for tablet 641a0b1632204ce89ed10fa1ad6f8408 (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.548491   741 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48432:
table { table_name: "default.Table_5f1fb0758cf1413eaed1d8b4717aa902" } new_table_name: "default.Table_d9954c97f0054586ab7cf9e5e98b3fa4" modify_external_catalogs: true
I20261118 08:54:26.548903  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 641a0b1632204ce89ed10fa1ad6f8408. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.552098  1336 tablet_bootstrap.cc:492] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:26.552551  1336 ts_tablet_manager.cc:1449] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.079s	user 0.003s	sys 0.009s
I20261118 08:54:26.553936  1338 tablet_bootstrap.cc:492] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:26.555884  1337 ts_tablet_manager.cc:1480] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.053s	user 0.012s	sys 0.000s
W20261118 08:54:26.558378   729 catalog_manager.cc:5045] TS 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): delete failed for tablet 043a26a7d5004cc883eab2489a3ac5ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 043a26a7d5004cc883eab2489a3ac5ba already in progress: creating tablet
I20261118 08:54:26.563860  1338 ts_tablet_manager.cc:1449] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.081s	user 0.000s	sys 0.013s
I20261118 08:54:26.556705  1121 heartbeater.cc:500] Master 127.0.72.188:41259 was elected leader, sending a full tablet report...
I20261118 08:54:26.564761  1336 raft_consensus.cc:360] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.565192  1336 raft_consensus.cc:386] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.565330  1336 raft_consensus.cc:748] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.566435  1337 tablet_bootstrap.cc:492] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
W20261118 08:54:26.566852   730 catalog_manager.cc:5045] TS ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): delete failed for tablet 043a26a7d5004cc883eab2489a3ac5ba because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 043a26a7d5004cc883eab2489a3ac5ba already in progress: creating tablet
I20261118 08:54:26.568606  1336 consensus_queue.cc:261] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.572038  1342 tablet_replica.cc:333] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:26.572589  1342 raft_consensus.cc:2261] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:26.573064  1342 raft_consensus.cc:2290] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:26.574049  1337 tablet_bootstrap.cc:654] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.580785  1342 ts_tablet_manager.cc:1962] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:26.590091  1337 tablet_bootstrap.cc:492] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:26.590378  1337 ts_tablet_manager.cc:1449] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.024s	user 0.001s	sys 0.004s
I20261118 08:54:26.591488  1337 raft_consensus.cc:360] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.591763  1337 raft_consensus.cc:386] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.591907  1337 raft_consensus.cc:748] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.592355  1337 consensus_queue.cc:261] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.593730  1337 ts_tablet_manager.cc:1480] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 08:54:26.594102  1337 tablet_bootstrap.cc:492] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:26.595371  1338 raft_consensus.cc:360] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.596056  1337 tablet_bootstrap.cc:654] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
W20261118 08:54:26.597558  1124 tablet.cc:2441] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 08:54:26.603547  1338 raft_consensus.cc:386] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.603704  1338 raft_consensus.cc:748] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.604493  1338 consensus_queue.cc:261] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.608196  1336 ts_tablet_manager.cc:1480] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.055s	user 0.020s	sys 0.006s
W20261118 08:54:26.611773   986 tablet.cc:2441] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 08:54:26.611979   983 heartbeater.cc:500] Master 127.0.72.188:41259 was elected leader, sending a full tablet report...
I20261118 08:54:26.613361  1337 tablet_bootstrap.cc:492] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:26.613554  1337 ts_tablet_manager.cc:1449] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.000s
I20261118 08:54:26.614715  1337 raft_consensus.cc:360] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.614925  1337 raft_consensus.cc:386] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.615006  1337 raft_consensus.cc:748] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.615288  1337 consensus_queue.cc:261] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.616365  1338 ts_tablet_manager.cc:1480] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.000s
I20261118 08:54:26.616532  1337 ts_tablet_manager.cc:1480] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 08:54:26.616667  1256 heartbeater.cc:500] Master 127.0.72.188:41259 was elected leader, sending a full tablet report...
I20261118 08:54:26.616865  1337 tablet_bootstrap.cc:492] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:26.619091  1337 tablet_bootstrap.cc:654] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.622673  1338 tablet_bootstrap.cc:492] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:26.624017  1336 tablet_bootstrap.cc:492] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:26.624466  1338 tablet_bootstrap.cc:654] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.626225  1336 tablet_bootstrap.cc:654] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.635164  1336 tablet_bootstrap.cc:492] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:26.635505  1336 ts_tablet_manager.cc:1449] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.012s	user 0.003s	sys 0.003s
I20261118 08:54:26.635982  1337 tablet_bootstrap.cc:492] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:26.636225  1337 ts_tablet_manager.cc:1449] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.001s
I20261118 08:54:26.636746  1336 raft_consensus.cc:360] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.637014  1336 raft_consensus.cc:386] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.637019  1337 raft_consensus.cc:360] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.637149  1336 raft_consensus.cc:748] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.637306  1337 raft_consensus.cc:386] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.637449  1337 raft_consensus.cc:748] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.637465  1336 consensus_queue.cc:261] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.637740  1337 consensus_queue.cc:261] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.638027  1336 ts_tablet_manager.cc:1480] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 08:54:26.639055  1337 ts_tablet_manager.cc:1480] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.003s	user 0.000s	sys 0.001s
I20261118 08:54:26.641839  1338 tablet_bootstrap.cc:492] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:26.642074  1338 ts_tablet_manager.cc:1449] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.000s
I20261118 08:54:26.643150  1338 raft_consensus.cc:360] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.643435  1338 raft_consensus.cc:386] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.643594  1338 raft_consensus.cc:748] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.643900  1338 consensus_queue.cc:261] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.646380  1338 ts_tablet_manager.cc:1480] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 08:54:26.646910  1338 tablet_bootstrap.cc:492] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:26.648949  1338 tablet_bootstrap.cc:654] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.652192  1336 tablet_bootstrap.cc:492] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:26.654280  1336 tablet_bootstrap.cc:654] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.662907  1336 tablet_bootstrap.cc:492] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:26.663211  1336 ts_tablet_manager.cc:1449] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.011s	user 0.005s	sys 0.005s
I20261118 08:54:26.663223  1338 tablet_bootstrap.cc:492] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:26.663437  1338 ts_tablet_manager.cc:1449] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.017s	user 0.006s	sys 0.000s
I20261118 08:54:26.664147  1336 raft_consensus.cc:360] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.664285  1338 raft_consensus.cc:360] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.664433  1336 raft_consensus.cc:386] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.664554  1336 raft_consensus.cc:748] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.664572  1338 raft_consensus.cc:386] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.664661  1338 raft_consensus.cc:748] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.664928  1336 consensus_queue.cc:261] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.665016  1338 consensus_queue.cc:261] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.667377  1338 ts_tablet_manager.cc:1480] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.001s
I20261118 08:54:26.668778  1338 tablet_bootstrap.cc:492] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:26.670697  1338 tablet_bootstrap.cc:654] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.672175  1336 ts_tablet_manager.cc:1480] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.000s
I20261118 08:54:26.677323  1336 tablet_bootstrap.cc:492] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:26.679226  1336 tablet_bootstrap.cc:654] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.684708  1338 tablet_bootstrap.cc:492] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:26.684986  1338 ts_tablet_manager.cc:1449] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.002s
I20261118 08:54:26.685914  1338 raft_consensus.cc:360] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.686242  1338 raft_consensus.cc:386] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.686383  1338 raft_consensus.cc:748] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.686746  1338 consensus_queue.cc:261] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.690821  1338 ts_tablet_manager.cc:1480] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.001s
I20261118 08:54:26.696619   737 catalog_manager.cc:2215] created HMS catalog entry for table  [id=1950821cc4b84c97939b8da6baaab97a]
I20261118 08:54:26.701596  1336 tablet_bootstrap.cc:492] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:26.707162  1336 ts_tablet_manager.cc:1449] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.004s
I20261118 08:54:26.708218  1336 raft_consensus.cc:360] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.708500  1336 raft_consensus.cc:386] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.708647  1336 raft_consensus.cc:748] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.708950  1336 consensus_queue.cc:261] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.711335  1336 ts_tablet_manager.cc:1480] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 08:54:26.718924   918 tablet_service.cc:1500] Processing CreateTablet for tablet 3e292319f0c44e34b49ae6d579df2ecb (DEFAULT_TABLE table=default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.719245  1190 tablet_service.cc:1500] Processing CreateTablet for tablet 3e292319f0c44e34b49ae6d579df2ecb (DEFAULT_TABLE table=default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.719416   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e292319f0c44e34b49ae6d579df2ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.719707  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e292319f0c44e34b49ae6d579df2ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.724362  1056 tablet_service.cc:1500] Processing CreateTablet for tablet 3e292319f0c44e34b49ae6d579df2ecb (DEFAULT_TABLE table=default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:26.727682   917 tablet_service.cc:1547] Processing DeleteTablet for tablet 043a26a7d5004cc883eab2489a3ac5ba with delete_type TABLET_DATA_DELETED (replaced by tablet 641a0b1632204ce89ed10fa1ad6f8408) from {username='slave'} at 127.0.0.1:44724
I20261118 08:54:26.728070  1343 tablet_replica.cc:333] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:26.728155  1191 tablet_service.cc:1547] Processing DeleteTablet for tablet 043a26a7d5004cc883eab2489a3ac5ba with delete_type TABLET_DATA_DELETED (replaced by tablet 641a0b1632204ce89ed10fa1ad6f8408) from {username='slave'} at 127.0.0.1:59210
I20261118 08:54:26.728551  1344 tablet_replica.cc:333] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:26.728605  1343 raft_consensus.cc:2261] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:26.729055  1343 raft_consensus.cc:2290] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:26.729144  1344 raft_consensus.cc:2261] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:26.729202  1338 tablet_bootstrap.cc:492] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:26.729574  1344 raft_consensus.cc:2290] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:26.730993  1338 tablet_bootstrap.cc:654] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.731021  1343 ts_tablet_manager.cc:1962] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:26.733340  1344 ts_tablet_manager.cc:1962] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:26.739281  1336 tablet_bootstrap.cc:492] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:26.741262  1336 tablet_bootstrap.cc:654] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.756130   737 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_0327d29a2f19450b872c54c554d7654e"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:26.757633  1336 tablet_bootstrap.cc:492] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:26.757836  1336 ts_tablet_manager.cc:1449] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.002s
I20261118 08:54:26.758687  1336 raft_consensus.cc:360] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.758905  1336 raft_consensus.cc:386] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.758976  1336 raft_consensus.cc:748] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.759316  1336 consensus_queue.cc:261] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.759984  1336 ts_tablet_manager.cc:1480] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.002s	user 0.003s	sys 0.000s
I20261118 08:54:26.761982  1338 tablet_bootstrap.cc:492] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:26.762231  1338 ts_tablet_manager.cc:1449] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.007s
I20261118 08:54:26.763080  1338 raft_consensus.cc:360] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.763352  1338 raft_consensus.cc:386] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.763525  1338 raft_consensus.cc:748] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.763681  1346 raft_consensus.cc:499] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:26.763936  1338 consensus_queue.cc:261] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.763913  1346 raft_consensus.cc:521] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.765875  1338 ts_tablet_manager.cc:1480] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 08:54:26.766124  1346 leader_election.cc:290] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:26.774869  1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "274fd2bd8f844da4904afc02a1b15b95" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:26.775404  1076 raft_consensus.cc:2490] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:26.776202  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "274fd2bd8f844da4904afc02a1b15b95" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:26.776614  1211 raft_consensus.cc:2490] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:26.777374   873 leader_election.cc:304] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:26.778245  1346 raft_consensus.cc:2826] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:26.778388  1346 raft_consensus.cc:499] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:26.778475  1346 raft_consensus.cc:3096] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.791992  1361 raft_consensus.cc:499] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:26.792295  1361 raft_consensus.cc:521] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.793228  1361 leader_election.cc:290] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:26.795630  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3e292319f0c44e34b49ae6d579df2ecb" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:26.795881  1211 raft_consensus.cc:2490] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:26.797363   873 leader_election.cc:304] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
W20261118 08:54:26.797839   873 leader_election.cc:343] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb
I20261118 08:54:26.799099  1342 ts_tablet_manager.cc:1975] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 08:54:26.799279  1342 log.cc:1199] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/043a26a7d5004cc883eab2489a3ac5ba
I20261118 08:54:26.799754  1342 ts_tablet_manager.cc:1996] T 043a26a7d5004cc883eab2489a3ac5ba P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:26.803651  1346 raft_consensus.cc:521] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.804399  1346 leader_election.cc:290] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Requested vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:26.805439  1346 raft_consensus.cc:2826] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:26.805460  1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "274fd2bd8f844da4904afc02a1b15b95" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:26.805542  1346 raft_consensus.cc:499] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:26.805644  1346 raft_consensus.cc:3096] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.805673  1076 raft_consensus.cc:3096] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.808547  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e292319f0c44e34b49ae6d579df2ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:26.809036  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "274fd2bd8f844da4904afc02a1b15b95" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:26.809253  1211 raft_consensus.cc:3096] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
W20261118 08:54:26.839232  1259 tablet.cc:2441] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 08:54:26.847829  1345 raft_consensus.cc:499] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:26.848053  1345 raft_consensus.cc:521] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.849817  1345 leader_election.cc:290] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:26.858054  1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "641a0b1632204ce89ed10fa1ad6f8408" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:26.858443  1210 raft_consensus.cc:2490] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 0.
I20261118 08:54:26.858376   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "641a0b1632204ce89ed10fa1ad6f8408" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:26.858783   938 raft_consensus.cc:2490] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 0.
I20261118 08:54:26.859484  1012 leader_election.cc:304] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:26.859932  1345 raft_consensus.cc:2826] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:26.860069  1345 raft_consensus.cc:499] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:26.860174  1345 raft_consensus.cc:3096] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.877820  1344 ts_tablet_manager.cc:1975] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 08:54:26.878036  1344 log.cc:1199] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/043a26a7d5004cc883eab2489a3ac5ba
I20261118 08:54:26.878469  1344 ts_tablet_manager.cc:1996] T 043a26a7d5004cc883eab2489a3ac5ba P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:26.878851  1076 raft_consensus.cc:2490] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:26.879498  1346 raft_consensus.cc:521] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.880347  1346 leader_election.cc:290] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Requested vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:26.880785  1343 ts_tablet_manager.cc:1975] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 08:54:26.880930  1343 log.cc:1199] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/043a26a7d5004cc883eab2489a3ac5ba
I20261118 08:54:26.881358  1343 ts_tablet_manager.cc:1996] T 043a26a7d5004cc883eab2489a3ac5ba P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:26.882301   729 catalog_manager.cc:5065] TS 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): tablet 043a26a7d5004cc883eab2489a3ac5ba (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]) successfully deleted
I20261118 08:54:26.882730  1337 tablet_bootstrap.cc:492] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:26.879369  1211 raft_consensus.cc:2490] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:26.885074   873 leader_election.cc:304] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:26.885299  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3e292319f0c44e34b49ae6d579df2ecb" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:26.885406  1346 raft_consensus.cc:2826] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:26.885514  1211 raft_consensus.cc:3096] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.885566  1346 raft_consensus.cc:705] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Becoming Leader. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Running, Role: LEADER
I20261118 08:54:26.885658  1337 tablet_bootstrap.cc:654] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:26.886025  1346 consensus_queue.cc:238] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.888749  1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3e292319f0c44e34b49ae6d579df2ecb" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
W20261118 08:54:26.889840   873 leader_election.cc:343] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Illegal state: must be running to vote when last-logged opid is not known
I20261118 08:54:26.897421  1361 consensus_queue.cc:1049] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:26.898283  1361 consensus_queue.cc:1049] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:26.908226  1337 tablet_bootstrap.cc:492] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:26.908594  1337 ts_tablet_manager.cc:1449] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.007s
I20261118 08:54:26.909608  1337 raft_consensus.cc:360] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.910866  1337 raft_consensus.cc:386] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:26.911443  1337 raft_consensus.cc:748] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:26.911964  1337 consensus_queue.cc:261] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.921526  1337 ts_tablet_manager.cc:1480] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
W20261118 08:54:26.945636  1323 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:26.945983  1323 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:26.946097  1323 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20261118 08:54:26.946841  1349 raft_consensus.cc:499] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:26.947023  1349 raft_consensus.cc:521] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:26.948855  1349 leader_election.cc:290] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:26.953573   730 catalog_manager.cc:5065] TS ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): tablet 043a26a7d5004cc883eab2489a3ac5ba (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]) successfully deleted
I20261118 08:54:26.958019   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8b0f9c9d6eef4b66aaab3bcafd810360" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:26.958328   938 raft_consensus.cc:2490] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
W20261118 08:54:26.959751  1323 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
I20261118 08:54:26.959733  1345 raft_consensus.cc:521] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
W20261118 08:54:26.960029  1323 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:26.960170  1323 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:26.960276  1323 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:26.960263  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8b0f9c9d6eef4b66aaab3bcafd810360" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:26.960501  1075 raft_consensus.cc:2490] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:26.960566  1345 leader_election.cc:290] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 election: Requested vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:26.961002  1211 raft_consensus.cc:2490] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:26.962110  1147 leader_election.cc:304] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:26.962657   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "641a0b1632204ce89ed10fa1ad6f8408" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:26.962874   938 raft_consensus.cc:3096] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.963220  1349 raft_consensus.cc:2826] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:26.963327  1349 raft_consensus.cc:499] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:26.963428  1349 raft_consensus.cc:3096] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.966149   873 leader_election.cc:304] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 213b5f7a64e74b109a3a9387e5975042
I20261118 08:54:26.966236   729 catalog_manager.cc:5065] TS 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): tablet 043a26a7d5004cc883eab2489a3ac5ba (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]) successfully deleted
I20261118 08:54:26.966663  1366 raft_consensus.cc:2826] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:26.966799  1366 raft_consensus.cc:705] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Becoming Leader. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Running, Role: LEADER
I20261118 08:54:26.966898  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "641a0b1632204ce89ed10fa1ad6f8408" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:26.967068  1211 raft_consensus.cc:3096] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.967108  1366 consensus_queue.cc:238] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:26.970654  1075 raft_consensus.cc:3096] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:26.971096  1361 consensus_queue.cc:1049] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:26.982343  1323 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=41231
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:26.984645  1323 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:26.987053  1323 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:26.996796  1383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:26.997169  1384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:26.997388  1386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:26.997841  1323 server_base.cc:1061] running on GCE node
I20261118 08:54:26.998600  1323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:27.000273  1323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:27.001509  1323 hybrid_clock.cc:648] HybridClock initialized: now 1794992067001461 us; error 43 us; skew 500 ppm
I20261118 08:54:27.001868  1323 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:27.004151  1323 webserver.cc:577] Webserver started at http://127.0.72.190:41231/ using document root <none> and password file <none>
I20261118 08:54:27.004866  1323 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:27.004987  1323 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:27.012755  1323 fs_manager.cc:714] Time spent opening directory manager: real 0.006s	user 0.007s	sys 0.000s
I20261118 08:54:27.016367  1392 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:27.017938  1323 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20261118 08:54:27.018126  1323 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
uuid: "78d4026bc230421ea10b8c68a9ef7871"
format_stamp: "Formatted at 2026-11-18 08:54:21 on dist-test-slave-m8c8"
I20261118 08:54:27.018846  1323 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:27.047648  1323 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:27.048566  1323 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:27.048842  1323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:27.055239   938 raft_consensus.cc:2490] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
I20261118 08:54:27.055269  1349 raft_consensus.cc:521] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.056185  1349 leader_election.cc:290] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Requested vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:27.060043  1211 raft_consensus.cc:2490] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
I20261118 08:54:27.061026  1011 leader_election.cc:304] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68; no voters: 
I20261118 08:54:27.061384  1345 raft_consensus.cc:2826] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:27.061548  1345 raft_consensus.cc:705] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 1 LEADER]: Becoming Leader. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Running, Role: LEADER
I20261118 08:54:27.061483  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8b0f9c9d6eef4b66aaab3bcafd810360" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:27.061730  1075 raft_consensus.cc:3096] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:27.061991  1345 consensus_queue.cc:238] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.062443  1346 consensus_queue.cc:1049] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:27.065182   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8b0f9c9d6eef4b66aaab3bcafd810360" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:27.065426   938 raft_consensus.cc:3096] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:27.075802  1367 consensus_queue.cc:1049] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:27.076505  1369 consensus_queue.cc:1049] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:27.104101  1323 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42341
I20261118 08:54:27.104101  1447 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42341 every 8 connection(s)
I20261118 08:54:27.106247  1323 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
I20261118 08:54:27.113914   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 1323
I20261118 08:54:27.117868  1448 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:27.124105  1448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap starting.
I20261118 08:54:27.136402  1448 log.cc:826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:27.145597  1448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 08:54:27.146621  1448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap complete.
I20261118 08:54:27.151911  1448 raft_consensus.cc:360] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:27.152612  1448 raft_consensus.cc:748] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.153448  1075 raft_consensus.cc:2490] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:27.153451   938 raft_consensus.cc:2490] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:27.153380  1448 consensus_queue.cc:261] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:27.154173  1147 leader_election.cc:304] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:27.154736  1349 raft_consensus.cc:2826] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:27.154886  1349 raft_consensus.cc:705] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 1 LEADER]: Becoming Leader. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Running, Role: LEADER
I20261118 08:54:27.155301  1349 consensus_queue.cc:238] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.162909  1380 consensus_queue.cc:1049] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:27.163647  1380 consensus_queue.cc:1049] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:27.182951  1452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:27.183295  1452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:27.185138  1448 sys_catalog.cc:565] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:27.190968  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:27.191669  1257 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:27.191898  1257 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
I20261118 08:54:27.199574  1257 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
I20261118 08:54:27.207501   984 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:27.222476  1473 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:27.222584  1473 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: loading cluster ID for follower catalog manager: success
I20261118 08:54:27.227195  1473 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: acquiring CA information for follower catalog manager: success
I20261118 08:54:27.229449  1473 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:27.402906  1423 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 1 FOLLOWER]: Advancing to term 2
I20261118 08:54:27.452775   742 catalog_manager.cc:2215] created HMS catalog entry for table  [id=19b3ec381a984304badca62634eadd70]
I20261118 08:54:27.475085  1452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c7d9f52503f41e4bafe2821e6fd6af1. Latest consensus state: current_term: 2 leader_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:27.475358  1452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:27.476130  1482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:27.476672  1482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:27.492223  1485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:27.492712  1485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:27.502545   742 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48432 (request call id 24) took 1689 ms. Trace:
I20261118 08:54:27.503508  1478 mvcc.cc:204] Tried to move back new op lower bound from 7352287505241210880 to 7352287501465374720. Current Snapshot: MvccSnapshot[applied={T|T < 7352287505241210880 or (T in {7352287505241210880})}]
I20261118 08:54:27.502738   742 rpcz_store.cc:276] 1118 08:54:25.813356 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:25.813435 (+    79us) service_pool.cc:224] Handling call
1118 08:54:25.830190 (+ 16755us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 08:54:25.830611 (+   421us) catalog_manager.cc:2188] Created new table and tablet info
1118 08:54:27.452721 (+1622110us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 08:54:27.502250 (+ 49529us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 08:54:27.502367 (+   117us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 08:54:27.502491 (+   124us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 08:54:27.455538 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 08:54:27.455703 (+   165us) write_op.cc:432] Acquiring schema lock in shared mode
1118 08:54:27.455710 (+     7us) write_op.cc:435] Acquired schema lock
1118 08:54:27.455713 (+     3us) tablet.cc:670] Decoding operations
1118 08:54:27.455829 (+   116us) write_op.cc:620] Acquiring the partition lock for write op
1118 08:54:27.455843 (+    14us) write_op.cc:641] Partition lock acquired for write op
1118 08:54:27.455848 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 08:54:27.455952 (+   104us) tablet.cc:709] Row locks acquired
1118 08:54:27.455954 (+     2us) write_op.cc:260] PREPARE: finished
1118 08:54:27.455995 (+    41us) write_op.cc:270] Start()
1118 08:54:27.456024 (+    29us) write_op.cc:276] Timestamp: P: 1794992067455868 usec, L: 0
1118 08:54:27.456027 (+     3us) op_driver.cc:350] REPLICATION: starting
1118 08:54:27.457230 (+  1203us) log.cc:844] Serialized 432 byte log entry
1118 08:54:27.496144 (+ 38914us) op_driver.cc:466] REPLICATION: finished
1118 08:54:27.501635 (+  5491us) write_op.cc:301] APPLY: starting
1118 08:54:27.501652 (+    17us) tablet.cc:1402] starting BulkCheckPresence
1118 08:54:27.501697 (+    45us) tablet.cc:1405] finished BulkCheckPresence
1118 08:54:27.501769 (+    72us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 08:54:27.501913 (+   144us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 08:54:27.501948 (+    35us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 08:54:27.501956 (+     8us) write_op.cc:312] APPLY: finished
1118 08:54:27.502049 (+    93us) log.cc:844] Serialized 28 byte log entry
1118 08:54:27.502121 (+    72us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:27.502140 (+    19us) write_op.cc:454] Released schema lock
1118 08:54:27.502146 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":865777,"Hive Metastore.run_cpu_time_us":298,"Hive Metastore.run_wall_time_us":755878,"spinlock_wait_cycles":3968,"child_traces":[["op",{"apply.queue_time_us":5366,"apply.run_cpu_time_us":613,"apply.run_wall_time_us":612,"num_ops":2,"prepare.queue_time_us":1942,"prepare.run_cpu_time_us":907,"prepare.run_wall_time_us":3500,"raft.queue_time_us":45,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":314,"replication_time_us":40047,"thread_start_us":7921,"threads_started":2}]]}
I20261118 08:54:27.523298  1056 tablet_service.cc:1500] Processing CreateTablet for tablet d546b7f20a7f4d3b9e5612080df04be6 (DEFAULT_TABLE table=default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.523836  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d546b7f20a7f4d3b9e5612080df04be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.523823  1190 tablet_service.cc:1500] Processing CreateTablet for tablet d546b7f20a7f4d3b9e5612080df04be6 (DEFAULT_TABLE table=default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.524183   918 tablet_service.cc:1500] Processing CreateTablet for tablet d546b7f20a7f4d3b9e5612080df04be6 (DEFAULT_TABLE table=default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.524267  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d546b7f20a7f4d3b9e5612080df04be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.525101   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d546b7f20a7f4d3b9e5612080df04be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.529069   742 catalog_manager.cc:5840] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2789ff5b54489c84966cff78be4721 (127.0.72.129). New cstate: current_term: 1 leader_uuid: "ad2789ff5b54489c84966cff78be4721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } }
I20261118 08:54:27.539335   742 catalog_manager.cc:5840] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2789ff5b54489c84966cff78be4721 (127.0.72.129). New cstate: current_term: 1 leader_uuid: "ad2789ff5b54489c84966cff78be4721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } }
I20261118 08:54:27.540046   742 catalog_manager.cc:5840] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "3934fbc351ee4a99add35742e90dfa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } health_report { overall_health: HEALTHY } } }
I20261118 08:54:27.540973   742 catalog_manager.cc:5840] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130). New cstate: current_term: 1 leader_uuid: "213b5f7a64e74b109a3a9387e5975042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } }
I20261118 08:54:27.550392  1491 tablet_bootstrap.cc:492] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:27.552649  1491 tablet_bootstrap.cc:654] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.560104  1492 tablet_bootstrap.cc:492] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:27.560722  1493 tablet_bootstrap.cc:492] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:27.565335   742 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_8f6feb81c1374d42bc1e0acb856b75cd"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:27.565620  1491 tablet_bootstrap.cc:492] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:27.565769  1491 ts_tablet_manager.cc:1449] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.016s	user 0.003s	sys 0.003s
I20261118 08:54:27.566613  1491 raft_consensus.cc:360] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.566918  1491 raft_consensus.cc:386] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.567050  1491 raft_consensus.cc:748] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.567157  1493 tablet_bootstrap.cc:654] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.567444  1491 consensus_queue.cc:261] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.568045  1491 ts_tablet_manager.cc:1480] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20261118 08:54:27.568998  1492 tablet_bootstrap.cc:654] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.571044  1257 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:27.571219  1257 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
I20261118 08:54:27.575944  1410 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:53371; Asking this server to re-register.
I20261118 08:54:27.576886  1492 tablet_bootstrap.cc:492] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:27.576936  1493 tablet_bootstrap.cc:492] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:27.577065  1257 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:27.577090  1492 ts_tablet_manager.cc:1449] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.017s	user 0.004s	sys 0.000s
I20261118 08:54:27.577144  1493 ts_tablet_manager.cc:1449] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.017s	user 0.002s	sys 0.002s
I20261118 08:54:27.577914  1492 raft_consensus.cc:360] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.577925  1493 raft_consensus.cc:360] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.578097  1493 raft_consensus.cc:386] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.578105  1492 raft_consensus.cc:386] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.578205  1492 raft_consensus.cc:748] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.578240  1493 raft_consensus.cc:748] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.578433  1492 consensus_queue.cc:261] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.578974  1492 ts_tablet_manager.cc:1480] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 08:54:27.579208  1410 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:27.580797  1493 consensus_queue.cc:261] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.581216  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:27.582895  1493 ts_tablet_manager.cc:1480] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.001s
I20261118 08:54:27.584492   984 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:27.585582  1410 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:39561; Asking this server to re-register.
I20261118 08:54:27.586342  1122 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:27.588245  1411 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:46529; Asking this server to re-register.
I20261118 08:54:27.588325  1410 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:27.589033   984 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:27.590528  1410 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:27.664546   739 catalog_manager.cc:2215] created HMS catalog entry for table  [id=812c3949fc5544c09b17567b3e81977b]
I20261118 08:54:27.672937   739 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48432 (request call id 27) took 1859 ms. Trace:
I20261118 08:54:27.673100   739 rpcz_store.cc:276] 1118 08:54:25.813721 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:25.813768 (+    47us) service_pool.cc:224] Handling call
1118 08:54:25.831148 (+ 17380us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 08:54:25.831605 (+   457us) catalog_manager.cc:2188] Created new table and tablet info
1118 08:54:27.664479 (+1832874us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 08:54:27.672607 (+  8128us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 08:54:27.672754 (+   147us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 08:54:27.672885 (+   131us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 08:54:27.665525 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 08:54:27.665696 (+   171us) write_op.cc:432] Acquiring schema lock in shared mode
1118 08:54:27.665703 (+     7us) write_op.cc:435] Acquired schema lock
1118 08:54:27.665706 (+     3us) tablet.cc:670] Decoding operations
1118 08:54:27.665810 (+   104us) write_op.cc:620] Acquiring the partition lock for write op
1118 08:54:27.665825 (+    15us) write_op.cc:641] Partition lock acquired for write op
1118 08:54:27.665832 (+     7us) tablet.cc:693] Acquiring locks for 2 operations
1118 08:54:27.665933 (+   101us) tablet.cc:709] Row locks acquired
1118 08:54:27.665936 (+     3us) write_op.cc:260] PREPARE: finished
1118 08:54:27.665982 (+    46us) write_op.cc:270] Start()
1118 08:54:27.666030 (+    48us) write_op.cc:276] Timestamp: P: 1794992067665939 usec, L: 0
1118 08:54:27.666033 (+     3us) op_driver.cc:350] REPLICATION: starting
1118 08:54:27.666246 (+   213us) log.cc:844] Serialized 432 byte log entry
1118 08:54:27.669977 (+  3731us) op_driver.cc:466] REPLICATION: finished
1118 08:54:27.670075 (+    98us) write_op.cc:301] APPLY: starting
1118 08:54:27.670091 (+    16us) tablet.cc:1402] starting BulkCheckPresence
1118 08:54:27.670137 (+    46us) tablet.cc:1405] finished BulkCheckPresence
1118 08:54:27.670163 (+    26us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 08:54:27.670389 (+   226us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 08:54:27.670424 (+    35us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 08:54:27.670433 (+     9us) write_op.cc:312] APPLY: finished
1118 08:54:27.670522 (+    89us) log.cc:844] Serialized 28 byte log entry
1118 08:54:27.670588 (+    66us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:27.670609 (+    21us) write_op.cc:454] Released schema lock
1118 08:54:27.670616 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 08:54:27.670697 (+    81us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:27.670701 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1620705,"Hive Metastore.run_cpu_time_us":340,"Hive Metastore.run_wall_time_us":211953,"child_traces":[["op",{"apply.queue_time_us":55,"apply.run_cpu_time_us":612,"apply.run_wall_time_us":611,"mutex_wait_us":2525,"num_ops":2,"prepare.queue_time_us":92,"prepare.run_cpu_time_us":879,"prepare.run_wall_time_us":879,"raft.queue_time_us":122,"raft.run_cpu_time_us":729,"raft.run_wall_time_us":727,"replication_time_us":3914,"spinlock_wait_cycles":100992}]]}
I20261118 08:54:27.684930   918 tablet_service.cc:1500] Processing CreateTablet for tablet 8285b9829b7d4250a275063f36e1c93a (DEFAULT_TABLE table=default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.685309  1190 tablet_service.cc:1500] Processing CreateTablet for tablet 8285b9829b7d4250a275063f36e1c93a (DEFAULT_TABLE table=default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.685312  1056 tablet_service.cc:1500] Processing CreateTablet for tablet 8285b9829b7d4250a275063f36e1c93a (DEFAULT_TABLE table=default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.685442   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8285b9829b7d4250a275063f36e1c93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.685729  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8285b9829b7d4250a275063f36e1c93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.685768  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8285b9829b7d4250a275063f36e1c93a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.690616  1454 raft_consensus.cc:499] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:27.690796  1454 raft_consensus.cc:521] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.692180  1491 tablet_bootstrap.cc:492] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:27.692283  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d546b7f20a7f4d3b9e5612080df04be6" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:27.692485  1075 raft_consensus.cc:2490] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:27.693302  1146 leader_election.cc:304] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68; no voters: 
I20261118 08:54:27.693877  1454 leader_election.cc:290] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:27.694052  1454 raft_consensus.cc:2826] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:27.694155  1454 raft_consensus.cc:499] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:27.694242  1491 tablet_bootstrap.cc:654] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.694907   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d546b7f20a7f4d3b9e5612080df04be6" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:27.695166   938 raft_consensus.cc:2490] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:27.694258  1454 raft_consensus.cc:3096] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:27.696564  1492 tablet_bootstrap.cc:492] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:27.698309  1492 tablet_bootstrap.cc:654] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.710462  1491 tablet_bootstrap.cc:492] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:27.710673  1491 ts_tablet_manager.cc:1449] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.001s
I20261118 08:54:27.711649  1491 raft_consensus.cc:360] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.711879  1491 raft_consensus.cc:386] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.711948  1491 raft_consensus.cc:748] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.712221  1491 consensus_queue.cc:261] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.712700  1491 ts_tablet_manager.cc:1480] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 08:54:27.712929  1493 tablet_bootstrap.cc:492] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:27.714751  1493 tablet_bootstrap.cc:654] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.725594  1492 tablet_bootstrap.cc:492] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:27.725852  1492 ts_tablet_manager.cc:1449] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.029s	user 0.002s	sys 0.017s
I20261118 08:54:27.726055   739 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_82d1541f550a4d5482cff56810c97a22"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:27.726696  1492 raft_consensus.cc:360] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.726918  1492 raft_consensus.cc:386] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.726989  1492 raft_consensus.cc:748] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.727262  1492 consensus_queue.cc:261] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.732569  1492 ts_tablet_manager.cc:1480] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.002s
I20261118 08:54:27.734016  1493 tablet_bootstrap.cc:492] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:27.734349  1493 ts_tablet_manager.cc:1449] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.000s
I20261118 08:54:27.735267  1493 raft_consensus.cc:360] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.735587  1493 raft_consensus.cc:386] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.735723  1493 raft_consensus.cc:748] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.736085  1493 consensus_queue.cc:261] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.739147  1493 ts_tablet_manager.cc:1480] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 08:54:27.762724  1454 raft_consensus.cc:521] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.763480  1454 leader_election.cc:290] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Requested vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:27.764048  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d546b7f20a7f4d3b9e5612080df04be6" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:27.764281  1075 raft_consensus.cc:3096] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:27.764261   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d546b7f20a7f4d3b9e5612080df04be6" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:27.764451   938 raft_consensus.cc:3096] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:27.767246  1075 raft_consensus.cc:2490] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:27.767246   938 raft_consensus.cc:2490] T d546b7f20a7f4d3b9e5612080df04be6 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:27.767839  1147 leader_election.cc:304] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:27.768180  1454 raft_consensus.cc:2826] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:27.768344  1454 raft_consensus.cc:705] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [term 1 LEADER]: Becoming Leader. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Running, Role: LEADER
I20261118 08:54:27.768659  1454 consensus_queue.cc:238] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.774053  1497 consensus_queue.cc:1049] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:27.774451  1497 consensus_queue.cc:1049] T d546b7f20a7f4d3b9e5612080df04be6 P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:27.865644   734 catalog_manager.cc:2215] created HMS catalog entry for table  [id=0146c51b5d004bd6b63f5e1c9e1dcf30]
I20261118 08:54:27.872510   734 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48432 (request call id 32) took 2056 ms. Trace:
I20261118 08:54:27.872675   734 rpcz_store.cc:276] 1118 08:54:25.815462 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:25.815528 (+    66us) service_pool.cc:224] Handling call
1118 08:54:25.832166 (+ 16638us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 08:54:25.832580 (+   414us) catalog_manager.cc:2188] Created new table and tablet info
1118 08:54:27.865559 (+2032979us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 08:54:27.871669 (+  6110us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 08:54:27.871797 (+   128us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 08:54:27.871918 (+   121us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 08:54:27.866413 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 08:54:27.866550 (+   137us) write_op.cc:432] Acquiring schema lock in shared mode
1118 08:54:27.866556 (+     6us) write_op.cc:435] Acquired schema lock
1118 08:54:27.866559 (+     3us) tablet.cc:670] Decoding operations
1118 08:54:27.866657 (+    98us) write_op.cc:620] Acquiring the partition lock for write op
1118 08:54:27.866671 (+    14us) write_op.cc:641] Partition lock acquired for write op
1118 08:54:27.866677 (+     6us) tablet.cc:693] Acquiring locks for 2 operations
1118 08:54:27.866805 (+   128us) tablet.cc:709] Row locks acquired
1118 08:54:27.866807 (+     2us) write_op.cc:260] PREPARE: finished
1118 08:54:27.866850 (+    43us) write_op.cc:270] Start()
1118 08:54:27.866888 (+    38us) write_op.cc:276] Timestamp: P: 1794992067866768 usec, L: 0
1118 08:54:27.866892 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 08:54:27.867096 (+   204us) log.cc:844] Serialized 432 byte log entry
1118 08:54:27.871007 (+  3911us) op_driver.cc:466] REPLICATION: finished
1118 08:54:27.871117 (+   110us) write_op.cc:301] APPLY: starting
1118 08:54:27.871135 (+    18us) tablet.cc:1402] starting BulkCheckPresence
1118 08:54:27.871182 (+    47us) tablet.cc:1405] finished BulkCheckPresence
1118 08:54:27.871219 (+    37us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 08:54:27.871340 (+   121us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 08:54:27.871404 (+    64us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 08:54:27.871413 (+     9us) write_op.cc:312] APPLY: finished
1118 08:54:27.871492 (+    79us) log.cc:844] Serialized 28 byte log entry
1118 08:54:27.871554 (+    62us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:27.871575 (+    21us) write_op.cc:454] Released schema lock
1118 08:54:27.871582 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 08:54:27.871649 (+    67us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:27.871653 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1831784,"Hive Metastore.run_cpu_time_us":332,"Hive Metastore.run_wall_time_us":200977,"child_traces":[["op",{"apply.queue_time_us":62,"apply.run_cpu_time_us":523,"apply.run_wall_time_us":522,"num_ops":2,"prepare.queue_time_us":73,"prepare.run_cpu_time_us":816,"prepare.run_wall_time_us":817,"raft.queue_time_us":590,"raft.run_cpu_time_us":659,"raft.run_wall_time_us":657,"replication_time_us":4084}]]}
I20261118 08:54:27.883325  1190 tablet_service.cc:1500] Processing CreateTablet for tablet 39e7203496e0458687be52c3fadd15ea (DEFAULT_TABLE table=default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.883539   918 tablet_service.cc:1500] Processing CreateTablet for tablet 39e7203496e0458687be52c3fadd15ea (DEFAULT_TABLE table=default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.883710  1056 tablet_service.cc:1500] Processing CreateTablet for tablet 39e7203496e0458687be52c3fadd15ea (DEFAULT_TABLE table=default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:27.883848  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e7203496e0458687be52c3fadd15ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.883984   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e7203496e0458687be52c3fadd15ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.884114  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39e7203496e0458687be52c3fadd15ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:27.890349  1491 tablet_bootstrap.cc:492] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:27.890872  1493 tablet_bootstrap.cc:492] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:27.892396  1491 tablet_bootstrap.cc:654] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.893070  1493 tablet_bootstrap.cc:654] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.896503   734 catalog_manager.cc:5840] T d546b7f20a7f4d3b9e5612080df04be6 P 213b5f7a64e74b109a3a9387e5975042 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131). New cstate: current_term: 1 leader_uuid: "3934fbc351ee4a99add35742e90dfa68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } }
I20261118 08:54:27.901801  1491 tablet_bootstrap.cc:492] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:27.901988  1491 ts_tablet_manager.cc:1449] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.003s
I20261118 08:54:27.902786  1491 raft_consensus.cc:360] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.903002  1491 raft_consensus.cc:386] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.903102  1491 raft_consensus.cc:748] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.903354  1491 consensus_queue.cc:261] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.903939  1491 ts_tablet_manager.cc:1480] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 08:54:27.908757  1493 tablet_bootstrap.cc:492] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:27.908947  1493 ts_tablet_manager.cc:1449] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.005s
I20261118 08:54:27.909128   734 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48432:
table { table_name: "default.Table_1ee6cacf31104df284b253928039759b" } new_table_name: "default.Table_29f622723bbb4a47970de8920bf7d580" modify_external_catalogs: true
I20261118 08:54:27.909822  1493 raft_consensus.cc:360] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.910045  1493 raft_consensus.cc:386] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.910143  1493 raft_consensus.cc:748] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.911229  1493 consensus_queue.cc:261] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.913341  1493 ts_tablet_manager.cc:1480] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.001s
I20261118 08:54:27.914512  1492 tablet_bootstrap.cc:492] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:27.916465  1492 tablet_bootstrap.cc:654] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:27.919970  1492 tablet_bootstrap.cc:492] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:27.920140  1492 ts_tablet_manager.cc:1449] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.006s	user 0.005s	sys 0.000s
I20261118 08:54:27.920828  1492 raft_consensus.cc:360] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.920986  1492 raft_consensus.cc:386] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:27.921068  1492 raft_consensus.cc:748] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:27.921312  1492 consensus_queue.cc:261] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.923293  1492 ts_tablet_manager.cc:1480] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 08:54:27.943678  1509 raft_consensus.cc:499] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:27.943918  1509 raft_consensus.cc:521] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:27.944746  1509 leader_election.cc:290] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:27.945997  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8285b9829b7d4250a275063f36e1c93a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:27.946024  1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8285b9829b7d4250a275063f36e1c93a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:27.946249  1210 raft_consensus.cc:2490] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:27.946274  1075 raft_consensus.cc:2490] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:27.947023   873 leader_election.cc:304] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:27.947624  1509 raft_consensus.cc:2826] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:27.947746  1509 raft_consensus.cc:499] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:27.947844  1509 raft_consensus.cc:3096] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.019199  1509 raft_consensus.cc:521] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.020187  1509 leader_election.cc:290] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Requested vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:28.021335  1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8285b9829b7d4250a275063f36e1c93a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:28.021559  1210 raft_consensus.cc:3096] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.021826  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8285b9829b7d4250a275063f36e1c93a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:28.022025  1075 raft_consensus.cc:3096] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.024044  1075 raft_consensus.cc:2490] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:28.024519   873 leader_election.cc:304] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:28.024911  1509 raft_consensus.cc:2826] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:28.025055  1509 raft_consensus.cc:705] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Becoming Leader. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Running, Role: LEADER
I20261118 08:54:28.025384  1509 consensus_queue.cc:238] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.030257  1509 consensus_queue.cc:1049] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
08:54:28.079 [HMS - ERROR - pool-6-thread-2] (RetryingHMSHandler.java:201) AlreadyExistsException(message:Table hive.default.table_8b08d3e4ed824f1680377a6d5e7518a8 already exists)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1839)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1786)
	at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:2035)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
	at jdk.proxy2/jdk.proxy2.$Proxy27.create_table_with_environment_context(Unknown Source)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14800)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$create_table_with_environment_context.getResult(ThriftHiveMetastore.java:14784)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
	at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

W20261118 08:54:28.081851   738 catalog_manager.cc:2211] Already present: failed to create HMS catalog entry for table  [id=977eb4910cb14ba3a41f40644a618555]: failed to create Hive MetaStore table: TException - service has thrown: AlreadyExistsException(message=Table hive.default.table_8b08d3e4ed824f1680377a6d5e7518a8 already exists)
I20261118 08:54:28.082399   738 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48432 (request call id 28) took 2268 ms. Trace:
I20261118 08:54:28.082540   738 rpcz_store.cc:276] 1118 08:54:25.814151 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:25.814641 (+   490us) service_pool.cc:224] Handling call
1118 08:54:25.833084 (+ 18443us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 08:54:25.833548 (+   464us) catalog_manager.cc:2188] Created new table and tablet info
1118 08:54:28.082351 (+2248803us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":2031844,"Hive Metastore.run_cpu_time_us":990,"Hive Metastore.run_wall_time_us":216154}
I20261118 08:54:28.094864   738 catalog_manager.cc:5840] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad2789ff5b54489c84966cff78be4721 (127.0.72.129). New cstate: current_term: 1 leader_uuid: "ad2789ff5b54489c84966cff78be4721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } health_report { overall_health: UNKNOWN } } }
I20261118 08:54:28.122166  1210 raft_consensus.cc:2490] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:28.123993   738 master_service.cc:965] ReplaceTablet: received request to replace tablet 043a26a7d5004cc883eab2489a3ac5ba from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:28.124179  1510 consensus_queue.cc:1049] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:28.124475   738 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48432 (request call id 45) took 1380 ms. Trace:
I20261118 08:54:28.124589   738 rpcz_store.cc:276] 1118 08:54:26.744113 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:28.123891 (+1379778us) service_pool.cc:224] Handling call
1118 08:54:28.124436 (+   545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 08:54:28.125015   738 master_service.cc:965] ReplaceTablet: received request to replace tablet 641a0b1632204ce89ed10fa1ad6f8408 from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:28.140023   738 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: Sending DeleteTablet for 3 replicas of tablet 641a0b1632204ce89ed10fa1ad6f8408
I20261118 08:54:28.140522   738 catalog_manager.cc:6936] ReplaceTablet: tablet 641a0b1632204ce89ed10fa1ad6f8408 deleted and replaced by tablet 4a6c27574dfb45a1b736aad9c209669e
I20261118 08:54:28.140697   738 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48432 (request call id 46) took 1386 ms. Trace:
I20261118 08:54:28.140825   918 tablet_service.cc:1547] Processing DeleteTablet for tablet 641a0b1632204ce89ed10fa1ad6f8408 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a6c27574dfb45a1b736aad9c209669e) from {username='slave'} at 127.0.0.1:44724
I20261118 08:54:28.140998  1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 641a0b1632204ce89ed10fa1ad6f8408 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a6c27574dfb45a1b736aad9c209669e) from {username='slave'} at 127.0.0.1:52346
I20261118 08:54:28.140786   738 rpcz_store.cc:276] 1118 08:54:26.754023 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:28.124956 (+1370933us) service_pool.cc:224] Handling call
1118 08:54:28.139697 (+ 14741us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:28.139703 (+     6us) write_op.cc:454] Released schema lock
1118 08:54:28.140638 (+   935us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 08:54:28.129079 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 08:54:28.129235 (+   156us) write_op.cc:432] Acquiring schema lock in shared mode
1118 08:54:28.129242 (+     7us) write_op.cc:435] Acquired schema lock
1118 08:54:28.129244 (+     2us) tablet.cc:670] Decoding operations
1118 08:54:28.129373 (+   129us) write_op.cc:620] Acquiring the partition lock for write op
1118 08:54:28.129388 (+    15us) write_op.cc:641] Partition lock acquired for write op
1118 08:54:28.129394 (+     6us) tablet.cc:693] Acquiring locks for 2 operations
1118 08:54:28.129482 (+    88us) tablet.cc:709] Row locks acquired
1118 08:54:28.129485 (+     3us) write_op.cc:260] PREPARE: finished
1118 08:54:28.129523 (+    38us) write_op.cc:270] Start()
1118 08:54:28.129559 (+    36us) write_op.cc:276] Timestamp: P: 1794992068129481 usec, L: 0
1118 08:54:28.129563 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 08:54:28.129794 (+   231us) log.cc:844] Serialized 658 byte log entry
1118 08:54:28.137069 (+  7275us) op_driver.cc:466] REPLICATION: finished
1118 08:54:28.137179 (+   110us) write_op.cc:301] APPLY: starting
1118 08:54:28.137198 (+    19us) tablet.cc:1402] starting BulkCheckPresence
1118 08:54:28.137245 (+    47us) tablet.cc:1405] finished BulkCheckPresence
1118 08:54:28.137291 (+    46us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 08:54:28.137409 (+   118us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 08:54:28.137470 (+    61us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 08:54:28.137477 (+     7us) write_op.cc:312] APPLY: finished
1118 08:54:28.137561 (+    84us) log.cc:844] Serialized 28 byte log entry
1118 08:54:28.137627 (+    66us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:28.137649 (+    22us) write_op.cc:454] Released schema lock
1118 08:54:28.137656 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":51,"apply.run_cpu_time_us":535,"apply.run_wall_time_us":535,"num_ops":2,"prepare.queue_time_us":3004,"prepare.run_cpu_time_us":851,"prepare.run_wall_time_us":853,"raft.queue_time_us":519,"raft.run_cpu_time_us":561,"raft.run_wall_time_us":560,"replication_time_us":7473}]]}
I20261118 08:54:28.141782  1190 tablet_service.cc:1547] Processing DeleteTablet for tablet 641a0b1632204ce89ed10fa1ad6f8408 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a6c27574dfb45a1b736aad9c209669e) from {username='slave'} at 127.0.0.1:59210
I20261118 08:54:28.142937   738 master_service.cc:965] ReplaceTablet: received request to replace tablet 274fd2bd8f844da4904afc02a1b15b95 from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:28.152129   917 tablet_service.cc:1500] Processing CreateTablet for tablet 4a6c27574dfb45a1b736aad9c209669e (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.152640   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a6c27574dfb45a1b736aad9c209669e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.154605  1518 tablet_replica.cc:333] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:28.154810  1518 raft_consensus.cc:2261] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.155038  1518 raft_consensus.cc:2290] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.155725  1518 ts_tablet_manager.cc:1962] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.156180  1520 tablet_replica.cc:333] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:28.156330  1520 raft_consensus.cc:2261] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.156502  1520 raft_consensus.cc:2290] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.156965  1520 ts_tablet_manager.cc:1962] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.159461  1054 tablet_service.cc:1500] Processing CreateTablet for tablet 4a6c27574dfb45a1b736aad9c209669e (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.159759  1519 tablet_replica.cc:333] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:28.159974  1519 raft_consensus.cc:2261] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:28.160058  1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a6c27574dfb45a1b736aad9c209669e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.160552  1519 raft_consensus.cc:2290] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.161211  1519 ts_tablet_manager.cc:1962] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.163789  1190 tablet_service.cc:1500] Processing CreateTablet for tablet 4a6c27574dfb45a1b736aad9c209669e (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.167006  1492 tablet_bootstrap.cc:492] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:28.167438   738 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 6c7d9f52503f41e4bafe2821e6fd6af1: Sending DeleteTablet for 3 replicas of tablet 274fd2bd8f844da4904afc02a1b15b95
I20261118 08:54:28.168105   738 catalog_manager.cc:6936] ReplaceTablet: tablet 274fd2bd8f844da4904afc02a1b15b95 deleted and replaced by tablet d92e535f2bf44815b6c003b56dd25b79
I20261118 08:54:28.168291   738 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48432 (request call id 47) took 1412 ms. Trace:
I20261118 08:54:28.168735  1492 tablet_bootstrap.cc:654] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.170406  1056 tablet_service.cc:1547] Processing DeleteTablet for tablet 274fd2bd8f844da4904afc02a1b15b95 with delete_type TABLET_DATA_DELETED (replaced by tablet d92e535f2bf44815b6c003b56dd25b79) from {username='slave'} at 127.0.0.1:52346
I20261118 08:54:28.170516  1191 tablet_service.cc:1547] Processing DeleteTablet for tablet 274fd2bd8f844da4904afc02a1b15b95 with delete_type TABLET_DATA_DELETED (replaced by tablet d92e535f2bf44815b6c003b56dd25b79) from {username='slave'} at 127.0.0.1:59210
I20261118 08:54:28.168445   738 rpcz_store.cc:276] 1118 08:54:26.755978 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:28.142849 (+1386871us) service_pool.cc:224] Handling call
1118 08:54:28.168256 (+ 25407us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 08:54:28.145009 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 08:54:28.145128 (+   119us) write_op.cc:432] Acquiring schema lock in shared mode
1118 08:54:28.145134 (+     6us) write_op.cc:435] Acquired schema lock
1118 08:54:28.145136 (+     2us) tablet.cc:670] Decoding operations
1118 08:54:28.145269 (+   133us) write_op.cc:620] Acquiring the partition lock for write op
1118 08:54:28.145302 (+    33us) write_op.cc:641] Partition lock acquired for write op
1118 08:54:28.145307 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 08:54:28.145388 (+    81us) tablet.cc:709] Row locks acquired
1118 08:54:28.145391 (+     3us) write_op.cc:260] PREPARE: finished
1118 08:54:28.145423 (+    32us) write_op.cc:270] Start()
1118 08:54:28.145452 (+    29us) write_op.cc:276] Timestamp: P: 1794992068145379 usec, L: 0
1118 08:54:28.145455 (+     3us) op_driver.cc:350] REPLICATION: starting
1118 08:54:28.145643 (+   188us) log.cc:844] Serialized 658 byte log entry
1118 08:54:28.149315 (+  3672us) op_driver.cc:466] REPLICATION: finished
1118 08:54:28.149979 (+   664us) write_op.cc:301] APPLY: starting
1118 08:54:28.149989 (+    10us) tablet.cc:1402] starting BulkCheckPresence
1118 08:54:28.150030 (+    41us) tablet.cc:1405] finished BulkCheckPresence
1118 08:54:28.150051 (+    21us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 08:54:28.150134 (+    83us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 08:54:28.150186 (+    52us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 08:54:28.150192 (+     6us) write_op.cc:312] APPLY: finished
1118 08:54:28.150257 (+    65us) log.cc:844] Serialized 28 byte log entry
1118 08:54:28.150309 (+    52us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:28.150325 (+    16us) write_op.cc:454] Released schema lock
1118 08:54:28.150330 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":626,"apply.run_cpu_time_us":411,"apply.run_wall_time_us":412,"num_ops":2,"prepare.queue_time_us":1108,"prepare.run_cpu_time_us":723,"prepare.run_wall_time_us":722,"raft.queue_time_us":431,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":3835,"thread_start_us":7997,"threads_started":1}]]}
I20261118 08:54:28.171923   918 tablet_service.cc:1547] Processing DeleteTablet for tablet 274fd2bd8f844da4904afc02a1b15b95 with delete_type TABLET_DATA_DELETED (replaced by tablet d92e535f2bf44815b6c003b56dd25b79) from {username='slave'} at 127.0.0.1:44724
I20261118 08:54:28.174630  1493 tablet_bootstrap.cc:492] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:28.175695   738 master_service.cc:965] ReplaceTablet: received request to replace tablet 043a26a7d5004cc883eab2489a3ac5ba from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:28.176476  1493 tablet_bootstrap.cc:654] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.177129   738 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48432 (request call id 48) took 1421 ms. Trace:
I20261118 08:54:28.177230  1518 ts_tablet_manager.cc:1975] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.177234   738 rpcz_store.cc:276] 1118 08:54:26.756093 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:28.175615 (+1419522us) service_pool.cc:224] Handling call
1118 08:54:28.176892 (+  1277us) spinlock_profiling.cc:245] Waited 921 us on lock 0x616000041b54. stack: 00007fa1036fb3a6 00007fa1036fb160 00007fa1026ec34d 00007fa119cbf704 00007fa119cbf6cd 00007fa112c1182d 00007fa112c11923 00007fa10f77b221 00007fa11767050d 00007fa11780c6fb 00007fa10e114e67 00007fa10e114b95 00007fa10a8438de 00007fa10a8427be 00007fa10a8489f8 00007fa10a84935f
1118 08:54:28.177071 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2118528}
I20261118 08:54:28.177429  1518 log.cc:1199] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/641a0b1632204ce89ed10fa1ad6f8408
I20261118 08:54:28.177527   738 master_service.cc:965] ReplaceTablet: received request to replace tablet 641a0b1632204ce89ed10fa1ad6f8408 from {username='slave'} at 127.0.0.1:48432
I20261118 08:54:28.178197   738 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48432 (request call id 49) took 1421 ms. Trace:
I20261118 08:54:28.178303   738 rpcz_store.cc:276] 1118 08:54:26.756228 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:28.177449 (+1421221us) service_pool.cc:224] Handling call
1118 08:54:28.178164 (+   715us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 08:54:28.178840  1518 ts_tablet_manager.cc:1996] T 641a0b1632204ce89ed10fa1ad6f8408 P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:28.179606  1492 tablet_bootstrap.cc:492] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:28.179767  1492 ts_tablet_manager.cc:1449] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.013s	user 0.006s	sys 0.000s
I20261118 08:54:28.179831  1518 tablet_replica.cc:333] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:28.180027  1518 raft_consensus.cc:2261] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:28.180552  1518 raft_consensus.cc:2290] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.181121  1518 ts_tablet_manager.cc:1962] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.180882  1492 raft_consensus.cc:360] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.181602  1493 tablet_bootstrap.cc:492] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:28.181710  1492 raft_consensus.cc:386] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.181756  1493 ts_tablet_manager.cc:1449] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.007s	user 0.000s	sys 0.004s
I20261118 08:54:28.181795  1492 raft_consensus.cc:748] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.182077  1492 consensus_queue.cc:261] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.183019  1520 ts_tablet_manager.cc:1975] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.183153  1520 log.cc:1199] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/641a0b1632204ce89ed10fa1ad6f8408
I20261118 08:54:28.183568  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a6c27574dfb45a1b736aad9c209669e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.183632  1520 ts_tablet_manager.cc:1996] T 641a0b1632204ce89ed10fa1ad6f8408 P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:28.184224  1191 tablet_service.cc:1500] Processing CreateTablet for tablet d92e535f2bf44815b6c003b56dd25b79 (DEFAULT_TABLE table=default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.184623  1520 tablet_replica.cc:333] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:28.184652  1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d92e535f2bf44815b6c003b56dd25b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.185087  1519 ts_tablet_manager.cc:1975] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.185187  1519 log.cc:1199] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/641a0b1632204ce89ed10fa1ad6f8408
I20261118 08:54:28.185590  1519 ts_tablet_manager.cc:1996] T 641a0b1632204ce89ed10fa1ad6f8408 P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:28.186549  1519 tablet_replica.cc:333] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:28.186731  1519 raft_consensus.cc:2261] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.186797   729 catalog_manager.cc:5065] TS 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): tablet 641a0b1632204ce89ed10fa1ad6f8408 (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]) successfully deleted
I20261118 08:54:28.186899  1519 raft_consensus.cc:2290] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.187322   729 catalog_manager.cc:5065] TS 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): tablet 641a0b1632204ce89ed10fa1ad6f8408 (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]) successfully deleted
I20261118 08:54:28.187465  1519 ts_tablet_manager.cc:1962] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.187819  1054 tablet_service.cc:1500] Processing CreateTablet for tablet d92e535f2bf44815b6c003b56dd25b79 (DEFAULT_TABLE table=default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.188201  1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d92e535f2bf44815b6c003b56dd25b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.190033  1491 tablet_bootstrap.cc:492] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:28.184827  1520 raft_consensus.cc:2261] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.191581  1520 raft_consensus.cc:2290] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.191668  1492 ts_tablet_manager.cc:1480] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.000s
I20261118 08:54:28.191924  1491 tablet_bootstrap.cc:654] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.192224  1520 ts_tablet_manager.cc:1962] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.195540  1491 tablet_bootstrap.cc:492] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:28.195708  1491 ts_tablet_manager.cc:1449] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20261118 08:54:28.192262   738 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48432:
name: "default.Table_903cefd8f2a74c89be4bf41b84d26ba4"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:28.196487  1491 raft_consensus.cc:360] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.196684  1491 raft_consensus.cc:386] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.196754  1491 raft_consensus.cc:748] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.197453  1491 consensus_queue.cc:261] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.198319  1520 ts_tablet_manager.cc:1975] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.198464  1520 log.cc:1199] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/274fd2bd8f844da4904afc02a1b15b95
I20261118 08:54:28.198895  1520 ts_tablet_manager.cc:1996] T 274fd2bd8f844da4904afc02a1b15b95 P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:28.199752  1491 ts_tablet_manager.cc:1480] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 08:54:28.200069  1491 tablet_bootstrap.cc:492] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:28.200425  1492 tablet_bootstrap.cc:492] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:28.202378  1492 tablet_bootstrap.cc:654] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.202404  1491 tablet_bootstrap.cc:654] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.204087   730 catalog_manager.cc:5065] TS ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): tablet 641a0b1632204ce89ed10fa1ad6f8408 (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]) successfully deleted
I20261118 08:54:28.205286  1493 raft_consensus.cc:360] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.205479  1493 raft_consensus.cc:386] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.205538  1493 raft_consensus.cc:748] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.205804  1493 consensus_queue.cc:261] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.206096   740 catalog_manager.cc:2215] created HMS catalog entry for table  [id=fa8c2c5f6be14649a1bd9abaca0167c8]
I20261118 08:54:28.206244  1519 ts_tablet_manager.cc:1975] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.206388  1519 log.cc:1199] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/274fd2bd8f844da4904afc02a1b15b95
I20261118 08:54:28.207355   729 catalog_manager.cc:5065] TS 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): tablet 274fd2bd8f844da4904afc02a1b15b95 (table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]) successfully deleted
I20261118 08:54:28.207969  1519 ts_tablet_manager.cc:1996] T 274fd2bd8f844da4904afc02a1b15b95 P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:28.209700   729 catalog_manager.cc:5065] TS 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): tablet 274fd2bd8f844da4904afc02a1b15b95 (table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]) successfully deleted
I20261118 08:54:28.209659   917 tablet_service.cc:1500] Processing CreateTablet for tablet d92e535f2bf44815b6c003b56dd25b79 (DEFAULT_TABLE table=default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.210067   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d92e535f2bf44815b6c003b56dd25b79. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.210307  1493 ts_tablet_manager.cc:1480] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.028s	user 0.000s	sys 0.002s
I20261118 08:54:28.217330  1493 tablet_bootstrap.cc:492] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:28.218408  1518 ts_tablet_manager.cc:1975] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.219112  1518 log.cc:1199] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/274fd2bd8f844da4904afc02a1b15b95
I20261118 08:54:28.219676  1518 ts_tablet_manager.cc:1996] T 274fd2bd8f844da4904afc02a1b15b95 P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:28.219807  1493 tablet_bootstrap.cc:654] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.220896   730 catalog_manager.cc:5065] TS ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): tablet 274fd2bd8f844da4904afc02a1b15b95 (table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]) successfully deleted
I20261118 08:54:28.221029   740 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48432 (request call id 26) took 2407 ms. Trace:
I20261118 08:54:28.221148   740 rpcz_store.cc:276] 1118 08:54:25.813605 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:25.813693 (+    88us) service_pool.cc:224] Handling call
1118 08:54:25.834062 (+ 20369us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 08:54:25.834455 (+   393us) catalog_manager.cc:2188] Created new table and tablet info
1118 08:54:28.206043 (+2371588us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 08:54:28.220729 (+ 14686us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 08:54:28.220872 (+   143us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 08:54:28.220987 (+   115us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 08:54:28.206851 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 08:54:28.206978 (+   127us) write_op.cc:432] Acquiring schema lock in shared mode
1118 08:54:28.206985 (+     7us) write_op.cc:435] Acquired schema lock
1118 08:54:28.206987 (+     2us) tablet.cc:670] Decoding operations
1118 08:54:28.207083 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
1118 08:54:28.207097 (+    14us) write_op.cc:641] Partition lock acquired for write op
1118 08:54:28.207101 (+     4us) tablet.cc:693] Acquiring locks for 2 operations
1118 08:54:28.207187 (+    86us) tablet.cc:709] Row locks acquired
1118 08:54:28.207189 (+     2us) write_op.cc:260] PREPARE: finished
1118 08:54:28.207240 (+    51us) write_op.cc:270] Start()
1118 08:54:28.207277 (+    37us) write_op.cc:276] Timestamp: P: 1794992068207180 usec, L: 0
1118 08:54:28.207281 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 08:54:28.207526 (+   245us) log.cc:844] Serialized 432 byte log entry
1118 08:54:28.216598 (+  9072us) op_driver.cc:466] REPLICATION: finished
1118 08:54:28.216744 (+   146us) write_op.cc:301] APPLY: starting
1118 08:54:28.216765 (+    21us) tablet.cc:1402] starting BulkCheckPresence
1118 08:54:28.216820 (+    55us) tablet.cc:1405] finished BulkCheckPresence
1118 08:54:28.216875 (+    55us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 08:54:28.217001 (+   126us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 08:54:28.217040 (+    39us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 08:54:28.217048 (+     8us) write_op.cc:312] APPLY: finished
1118 08:54:28.217144 (+    96us) log.cc:844] Serialized 28 byte log entry
1118 08:54:28.217217 (+    73us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:28.217242 (+    25us) write_op.cc:454] Released schema lock
1118 08:54:28.217250 (+     8us) write_op.cc:341] FINISH: Updating metrics
1118 08:54:28.217328 (+    78us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:28.217332 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":2247138,"Hive Metastore.run_cpu_time_us":288,"Hive Metastore.run_wall_time_us":124254,"child_traces":[["op",{"apply.queue_time_us":90,"apply.run_cpu_time_us":574,"apply.run_wall_time_us":572,"num_ops":2,"prepare.queue_time_us":62,"prepare.run_cpu_time_us":799,"prepare.run_wall_time_us":799,"raft.queue_time_us":3815,"raft.run_cpu_time_us":553,"raft.run_wall_time_us":553,"replication_time_us":9279}]]}
I20261118 08:54:28.227355  1492 tablet_bootstrap.cc:492] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:28.227538  1492 ts_tablet_manager.cc:1449] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.000s
I20261118 08:54:28.227716  1491 tablet_bootstrap.cc:492] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:28.227869  1491 ts_tablet_manager.cc:1449] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.004s
I20261118 08:54:28.228314  1492 raft_consensus.cc:360] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.228502  1492 raft_consensus.cc:386] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.228582  1492 raft_consensus.cc:748] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.228645  1491 raft_consensus.cc:360] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.228943  1491 raft_consensus.cc:386] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.229043  1491 raft_consensus.cc:748] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.229279  1491 consensus_queue.cc:261] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.231776   740 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48432:
table { table_name: "default.Table_ef26d2c593994e118ff3d01a0f5a9a1d" } new_table_name: "default.Table_0d932fab32a1414582d1f8370b6e2b23" modify_external_catalogs: true
I20261118 08:54:28.232607  1491 ts_tablet_manager.cc:1480] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.002s
I20261118 08:54:28.233337  1190 tablet_service.cc:1500] Processing CreateTablet for tablet a810b27a15594c708bfed8f4a5129912 (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.233852  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a810b27a15594c708bfed8f4a5129912. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.233889   917 tablet_service.cc:1500] Processing CreateTablet for tablet a810b27a15594c708bfed8f4a5129912 (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.228813  1492 consensus_queue.cc:261] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.233160  1054 tablet_service.cc:1500] Processing CreateTablet for tablet a810b27a15594c708bfed8f4a5129912 (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.234294   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a810b27a15594c708bfed8f4a5129912. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.234439  1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a810b27a15594c708bfed8f4a5129912. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.235597  1454 raft_consensus.cc:499] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:28.236354  1454 raft_consensus.cc:521] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.237030  1454 leader_election.cc:290] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:28.237749  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "39e7203496e0458687be52c3fadd15ea" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:28.237957  1075 raft_consensus.cc:2490] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:28.238700   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "39e7203496e0458687be52c3fadd15ea" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:28.238946   938 raft_consensus.cc:2490] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:28.239482  1147 leader_election.cc:304] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:28.240729  1454 raft_consensus.cc:2826] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:28.240823  1454 raft_consensus.cc:499] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:28.240921  1454 raft_consensus.cc:3096] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.243543  1492 ts_tablet_manager.cc:1480] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.000s
I20261118 08:54:28.245167  1493 tablet_bootstrap.cc:492] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:28.245335  1493 ts_tablet_manager.cc:1449] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.002s
I20261118 08:54:28.245249  1454 raft_consensus.cc:521] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.246146  1493 raft_consensus.cc:360] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.246351  1493 raft_consensus.cc:386] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.246366   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "39e7203496e0458687be52c3fadd15ea" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:28.246592   938 raft_consensus.cc:3096] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.246668  1492 tablet_bootstrap.cc:492] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:28.248759  1454 leader_election.cc:290] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Requested vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:28.249338   938 raft_consensus.cc:2490] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:28.249993  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "39e7203496e0458687be52c3fadd15ea" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:28.250187  1075 raft_consensus.cc:3096] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.250285  1147 leader_election.cc:304] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:28.250734  1454 raft_consensus.cc:2826] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:28.250932  1454 raft_consensus.cc:705] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [term 1 LEADER]: Becoming Leader. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Running, Role: LEADER
I20261118 08:54:28.251276  1454 consensus_queue.cc:238] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.253623  1075 raft_consensus.cc:2490] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:28.254199  1491 tablet_bootstrap.cc:492] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:28.256068  1491 tablet_bootstrap.cc:654] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.258052  1491 tablet_bootstrap.cc:492] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:28.258232  1491 ts_tablet_manager.cc:1449] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20261118 08:54:28.259035  1491 raft_consensus.cc:360] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.259244  1491 raft_consensus.cc:386] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.259326  1491 raft_consensus.cc:748] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.246452  1493 raft_consensus.cc:748] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.259644  1491 consensus_queue.cc:261] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.259927  1493 consensus_queue.cc:261] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.260182  1491 ts_tablet_manager.cc:1480] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.002s
I20261118 08:54:28.263559  1493 ts_tablet_manager.cc:1480] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.001s
I20261118 08:54:28.263891  1493 tablet_bootstrap.cc:492] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:28.265749  1493 tablet_bootstrap.cc:654] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.267858  1492 tablet_bootstrap.cc:654] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.271222  1502 consensus_queue.cc:1049] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:28.274747  1493 tablet_bootstrap.cc:492] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:28.274937  1493 ts_tablet_manager.cc:1449] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.011s	user 0.003s	sys 0.000s
I20261118 08:54:28.275234  1502 consensus_queue.cc:1049] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:28.275804  1493 raft_consensus.cc:360] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.277097  1493 raft_consensus.cc:386] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.277186  1493 raft_consensus.cc:748] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.277467  1493 consensus_queue.cc:261] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.278142  1493 ts_tablet_manager.cc:1480] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 08:54:28.279613   736 catalog_manager.cc:2215] created HMS catalog entry for table  [id=39ea4f28a4744d1290fe1e9246789b0a]
I20261118 08:54:28.285796   736 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48432 (request call id 30) took 2471 ms. Trace:
I20261118 08:54:28.285941   736 rpcz_store.cc:276] 1118 08:54:25.814632 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 08:54:25.814896 (+   264us) service_pool.cc:224] Handling call
1118 08:54:25.834997 (+ 20101us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 08:54:25.835465 (+   468us) catalog_manager.cc:2188] Created new table and tablet info
1118 08:54:28.279564 (+2444099us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 08:54:28.285509 (+  5945us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 08:54:28.285626 (+   117us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 08:54:28.285744 (+   118us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 08:54:28.280414 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 08:54:28.280549 (+   135us) write_op.cc:432] Acquiring schema lock in shared mode
1118 08:54:28.280555 (+     6us) write_op.cc:435] Acquired schema lock
1118 08:54:28.280557 (+     2us) tablet.cc:670] Decoding operations
1118 08:54:28.280650 (+    93us) write_op.cc:620] Acquiring the partition lock for write op
1118 08:54:28.280665 (+    15us) write_op.cc:641] Partition lock acquired for write op
1118 08:54:28.280670 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 08:54:28.280755 (+    85us) tablet.cc:709] Row locks acquired
1118 08:54:28.280758 (+     3us) write_op.cc:260] PREPARE: finished
1118 08:54:28.280796 (+    38us) write_op.cc:270] Start()
1118 08:54:28.280834 (+    38us) write_op.cc:276] Timestamp: P: 1794992068280723 usec, L: 0
1118 08:54:28.280838 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 08:54:28.281009 (+   171us) log.cc:844] Serialized 432 byte log entry
1118 08:54:28.284812 (+  3803us) op_driver.cc:466] REPLICATION: finished
1118 08:54:28.284944 (+   132us) write_op.cc:301] APPLY: starting
1118 08:54:28.284963 (+    19us) tablet.cc:1402] starting BulkCheckPresence
1118 08:54:28.285037 (+    74us) tablet.cc:1405] finished BulkCheckPresence
1118 08:54:28.285066 (+    29us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 08:54:28.285185 (+   119us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 08:54:28.285222 (+    37us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 08:54:28.285231 (+     9us) write_op.cc:312] APPLY: finished
1118 08:54:28.285317 (+    86us) log.cc:844] Serialized 28 byte log entry
1118 08:54:28.285390 (+    73us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:28.285414 (+    24us) write_op.cc:454] Released schema lock
1118 08:54:28.285421 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 08:54:28.285501 (+    80us) write_op.cc:489] Releasing partition, row and schema locks
1118 08:54:28.285505 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":2370456,"Hive Metastore.run_cpu_time_us":277,"Hive Metastore.run_wall_time_us":73189,"child_traces":[["op",{"apply.queue_time_us":81,"apply.run_cpu_time_us":547,"apply.run_wall_time_us":547,"num_ops":2,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":737,"prepare.run_wall_time_us":738,"raft.queue_time_us":469,"raft.run_cpu_time_us":464,"raft.run_wall_time_us":463,"replication_time_us":3942}]]}
I20261118 08:54:28.293460   736 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48432:
table { table_name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640" } modify_external_catalogs: true
I20261118 08:54:28.293871   736 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48432:
table { table_name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640" } modify_external_catalogs: true
F20261118 08:54:28.299798   823 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_bc1f66261ac64a87912e19a32ce0ca44, message={"server":"","servicePrincipal":"","db":"default","table":"table_bc1f66261ac64a87912e19a32ce0ca44","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_bc1f66261ac64a87912e19a32ce0ca44\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_bc1f66261ac64a87912e19a32ce0ca44\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_bc1f66261ac64a87912e19a32ce0ca44\",\"kudu.table_id\":\"4fcf984156584e30aac26283f71adda2\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_8b08d3e4ed824f1680377a6d5e7518a8, message={"server":"","servicePrincipal":"","db":"default","table":"table_8b08d3e4ed824f1680377a6d5e7518a8","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_8b08d3e4ed824f1680377a6d5e7518a8\",\"kudu.table_id\":\"b42eb739941947f8b7d79fddd923eb52\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20261118 08:54:28.305480   917 tablet_service.cc:1500] Processing CreateTablet for tablet 543333f1b0df4517a424aa536902aa29 (DEFAULT_TABLE table=default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.305960   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543333f1b0df4517a424aa536902aa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.306023  1190 tablet_service.cc:1500] Processing CreateTablet for tablet 543333f1b0df4517a424aa536902aa29 (DEFAULT_TABLE table=default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.306418  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543333f1b0df4517a424aa536902aa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.308720  1054 tablet_service.cc:1500] Processing CreateTablet for tablet 543333f1b0df4517a424aa536902aa29 (DEFAULT_TABLE table=default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.309131  1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543333f1b0df4517a424aa536902aa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.314401  1491 tablet_bootstrap.cc:492] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
    @     0x7fa1022d9d5d  google::LogMessage::Fail() at ??:0
I20261118 08:54:28.317199  1491 tablet_bootstrap.cc:654] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.319558  1345 raft_consensus.cc:499] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:28.319738  1345 raft_consensus.cc:521] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.320428  1345 leader_election.cc:290] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:28.321115   937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4a6c27574dfb45a1b736aad9c209669e" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:28.321344   937 raft_consensus.cc:2490] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 0.
I20261118 08:54:28.321985  1012 leader_election.cc:304] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
    @     0x7fa1022ddb23  google::LogMessage::SendToLog() at ??:0
I20261118 08:54:28.325440  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4a6c27574dfb45a1b736aad9c209669e" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:28.325652  1211 raft_consensus.cc:2490] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 0.
I20261118 08:54:28.326416  1345 raft_consensus.cc:2826] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:28.326537  1345 raft_consensus.cc:499] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:28.326642  1345 raft_consensus.cc:3096] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.329778  1493 tablet_bootstrap.cc:492] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:28.329692  1345 raft_consensus.cc:521] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.330845   937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4a6c27574dfb45a1b736aad9c209669e" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:28.331046   937 raft_consensus.cc:3096] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.331665  1493 tablet_bootstrap.cc:654] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
    @     0x7fa1022d975c  google::LogMessage::Flush() at ??:0
    @     0x7fa1022d99a1  google::LogMessage::~LogMessage() at ??:0
I20261118 08:54:28.334388   937 raft_consensus.cc:2490] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
I20261118 08:54:28.344208  1012 leader_election.cc:304] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:28.348232  1493 tablet_bootstrap.cc:492] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:28.348404  1493 ts_tablet_manager.cc:1449] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.000s
I20261118 08:54:28.349267  1493 raft_consensus.cc:360] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.349460  1493 raft_consensus.cc:386] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.349541  1493 raft_consensus.cc:748] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.349860  1493 consensus_queue.cc:261] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.350554  1493 ts_tablet_manager.cc:1480] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.001s
    @     0x7fa1177599a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 08:54:28.353995  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4a6c27574dfb45a1b736aad9c209669e" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:28.354213  1211 raft_consensus.cc:3096] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.357362  1211 raft_consensus.cc:2490] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
I20261118 08:54:28.357738  1345 leader_election.cc:290] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 election: Requested vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
    @     0x7fa117758251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 08:54:28.358060  1345 raft_consensus.cc:2826] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:28.358201  1345 raft_consensus.cc:705] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 1 LEADER]: Becoming Leader. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Running, Role: LEADER
I20261118 08:54:28.358489  1345 consensus_queue.cc:238] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
    @     0x7fa11775fecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
    @     0x7fa11775fd49  std::_Function_handler<>::_M_invoke() at ??:0
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
    @     0x7fa103722dc9  kudu::Thread::SuperviseThread() at ??:0
    @     0x7fa106dfd6db  start_thread at ??:0
    @     0x7fa0ff86d61f  clone at ??:0
I20261118 08:54:28.382212  1345 consensus_queue.cc:1049] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 08:54:28.394656  1012 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.72.129:39287, user_credentials={real_user=slave}} blocked reactor thread for 50484.6us
I20261118 08:54:28.396574  1539 consensus_queue.cc:1049] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20261118 08:54:28.416657  1491 tablet_bootstrap.cc:492] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:28.416646  1492 tablet_bootstrap.cc:492] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:28.416805  1491 ts_tablet_manager.cc:1449] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.103s	user 0.016s	sys 0.029s
I20261118 08:54:28.416821  1492 ts_tablet_manager.cc:1449] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.170s	user 0.040s	sys 0.074s
I20261118 08:54:28.417572  1491 raft_consensus.cc:360] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.417557  1492 raft_consensus.cc:360] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.417738  1491 raft_consensus.cc:386] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.417750  1492 raft_consensus.cc:386] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.417814  1492 raft_consensus.cc:748] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.417830  1491 raft_consensus.cc:748] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.418040  1492 consensus_queue.cc:261] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.418040  1491 consensus_queue.cc:261] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.418545  1492 ts_tablet_manager.cc:1480] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 08:54:28.418882  1492 tablet_bootstrap.cc:492] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:28.419857  1491 ts_tablet_manager.cc:1480] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.003s	user 0.001s	sys 0.001s
I20261118 08:54:28.422991  1492 tablet_bootstrap.cc:654] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.432242  1492 tablet_bootstrap.cc:492] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:28.432538  1492 ts_tablet_manager.cc:1449] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.000s
I20261118 08:54:28.433347  1492 raft_consensus.cc:360] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.433538  1492 raft_consensus.cc:386] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.433610  1492 raft_consensus.cc:748] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.433828  1492 consensus_queue.cc:261] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.434410  1492 ts_tablet_manager.cc:1480] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 08:54:28.527207  1539 raft_consensus.cc:499] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:28.527460  1539 raft_consensus.cc:521] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.528165  1539 leader_election.cc:290] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:28.528745   937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "543333f1b0df4517a424aa536902aa29" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:28.528935  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "543333f1b0df4517a424aa536902aa29" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:28.529016   937 raft_consensus.cc:2490] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 0.
I20261118 08:54:28.529121  1211 raft_consensus.cc:2490] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 0.
I20261118 08:54:28.529520  1012 leader_election.cc:304] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:28.529829  1539 raft_consensus.cc:2826] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:28.529918  1539 raft_consensus.cc:499] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:28.530020  1539 raft_consensus.cc:3096] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.531994  1539 raft_consensus.cc:521] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.532712  1539 leader_election.cc:290] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 election: Requested vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:28.533030   937 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "543333f1b0df4517a424aa536902aa29" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:28.533245   937 raft_consensus.cc:3096] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.533352  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "543333f1b0df4517a424aa536902aa29" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:28.533519  1211 raft_consensus.cc:3096] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.536197   937 raft_consensus.cc:2490] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
I20261118 08:54:28.536348  1211 raft_consensus.cc:2490] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
I20261118 08:54:28.536707  1012 leader_election.cc:304] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:28.537056  1539 raft_consensus.cc:2826] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:28.537232  1539 raft_consensus.cc:705] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 1 LEADER]: Becoming Leader. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Running, Role: LEADER
I20261118 08:54:28.537504  1539 consensus_queue.cc:238] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.541482  1345 consensus_queue.cc:1049] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:28.541908  1345 consensus_queue.cc:1049] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:28.571550  1510 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:28.571822  1510 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.572649  1510 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:28.573218  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d92e535f2bf44815b6c003b56dd25b79" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:28.573246  1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d92e535f2bf44815b6c003b56dd25b79" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:28.573457  1211 raft_consensus.cc:2490] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:28.573524  1076 raft_consensus.cc:2490] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:28.573940   873 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:28.574236  1510 raft_consensus.cc:2826] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:28.574329  1510 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:28.574433  1510 raft_consensus.cc:3096] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.576426  1510 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.577327  1510 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Requested vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:28.577734  1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d92e535f2bf44815b6c003b56dd25b79" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:28.577939  1076 raft_consensus.cc:3096] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.577891  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d92e535f2bf44815b6c003b56dd25b79" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:28.578084  1211 raft_consensus.cc:3096] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:28.580838  1211 raft_consensus.cc:2490] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:28.580852  1076 raft_consensus.cc:2490] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:28.581255   873 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:28.581540  1510 raft_consensus.cc:2826] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:28.582011  1510 raft_consensus.cc:705] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Becoming Leader. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Running, Role: LEADER
I20261118 08:54:28.582310  1510 consensus_queue.cc:238] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:28.586050  1510 consensus_queue.cc:1049] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:28.586565  1510 consensus_queue.cc:1049] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 08:54:28.618249  1263 connection.cc:573] client connection to 127.0.72.188:41259 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:28.619001  1263 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.188:41259)
W20261118 08:54:28.622732   871 connection.cc:573] client connection to 127.0.72.188:41259 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:28.625501  1263 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.188:41259)
W20261118 08:54:28.626159  1263 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.188:41259)
I20261118 08:54:28.618587  1323 master_runner.cc:187] Error getting master registration for 127.0.72.188:41259: Network error: recv got EOF from 127.0.72.188:41259 (error 108), 
I20261118 08:54:28.626886  1323 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:54:28.626922  1263 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.188:41259)
W20261118 08:54:28.636518  1567 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:28.623648 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.188:41259 (local address 127.0.72.129:43057)
1118 08:54:28.636280 (+ 12632us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:28.636466 (+   186us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":12485,"threads_started":1}
W20261118 08:54:28.640509  1566 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:28.622461 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.188:41259 (local address 127.0.72.131:49593)
1118 08:54:28.640246 (+ 17785us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:28.640438 (+   192us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":17652,"thread_start_us":17527,"threads_started":1}
W20261118 08:54:28.642477  1565 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:28.622080 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.188:41259 (local address 127.0.72.130:60761)
1118 08:54:28.642246 (+ 20166us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:28.642427 (+   181us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":20036,"thread_start_us":19969,"threads_started":1}
I20261118 08:54:28.801988  1569 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7d9f52503f41e4bafe2821e6fd6af1)
I20261118 08:54:28.802239  1569 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:28.803051  1570 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7d9f52503f41e4bafe2821e6fd6af1)
I20261118 08:54:28.803205  1570 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:28.803932  1570 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:28.804880  1569 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:28.805241   683 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca" is_pre_election: true
I20261118 08:54:28.805501   683 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 2.
W20261118 08:54:28.805843   657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111) [suppressed 4 similar messages]
I20261118 08:54:28.806042  1394 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 
I20261118 08:54:28.806595  1569 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20261118 08:54:28.806725  1569 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7d9f52503f41e4bafe2821e6fd6af1)
I20261118 08:54:28.806834  1569 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 2 FOLLOWER]: Advancing to term 3
W20261118 08:54:28.807274  1393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:28.808490   657 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:28.809656  1393 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:28.810768  1569 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:28.811672  1569 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 3 election: Requested vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:28.812171   683 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca"
I20261118 08:54:28.812387   683 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 2 FOLLOWER]: Advancing to term 3
W20261118 08:54:28.812678  1393 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:28.813727  1423 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "857a577b88ba46cc846ecd1a25a722ca" candidate_term: 3 candidate_status { last_received { term: 2 index: 33 } } ignore_live_leader: false dest_uuid: "78d4026bc230421ea10b8c68a9ef7871" is_pre_election: true
I20261118 08:54:28.814146  1423 raft_consensus.cc:2415] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 857a577b88ba46cc846ecd1a25a722ca in current term 3: Already voted for candidate 78d4026bc230421ea10b8c68a9ef7871 in this term.
I20261118 08:54:28.814743   658 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:28.815070   683 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 3.
I20261118 08:54:28.815229  1570 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20261118 08:54:28.815498  1394 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1
I20261118 08:54:28.815780  1569 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Leader election won for term 3
I20261118 08:54:28.816136  1569 raft_consensus.cc:705] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 LEADER]: Becoming Leader. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Running, Role: LEADER
I20261118 08:54:28.816545  1569 consensus_queue.cc:238] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 33, Committed index: 33, Last appended: 2.33, Last appended by leader: 33, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:28.817925  1576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 3 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:28.818229  1576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:28.819013  1579 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 08:54:28.819947   683 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Refusing update from remote peer 78d4026bc230421ea10b8c68a9ef7871: Log matching property violated. Preceding OpId in replica: term: 2 index: 33. Preceding OpId from leader: term: 3 index: 34. (index mismatch)
W20261118 08:54:28.820395  1393 consensus_peers.cc:597] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 -> Peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Couldn't send request to peer 6c7d9f52503f41e4bafe2821e6fd6af1. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:28.820622  1577 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 34, Last known committed idx: 33, Time since last communication: 0.000s
I20261118 08:54:28.822113  1570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 3 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:28.822363  1570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:28.822317  1579 catalog_manager.cc:694] Loaded metadata for table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30]
I20261118 08:54:28.822914  1579 catalog_manager.cc:694] Loaded metadata for table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a]
I20261118 08:54:28.823493  1579 catalog_manager.cc:694] Loaded metadata for table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70]
I20261118 08:54:28.823851  1569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:28.824137  1569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:28.824148  1579 catalog_manager.cc:694] Loaded metadata for table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]
I20261118 08:54:28.824548  1579 catalog_manager.cc:694] Loaded metadata for table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]
I20261118 08:54:28.824970  1579 catalog_manager.cc:694] Loaded metadata for table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]
I20261118 08:54:28.824920  1570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:28.825093  1570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:28.825384  1579 catalog_manager.cc:694] Loaded metadata for table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]
I20261118 08:54:28.825739  1579 catalog_manager.cc:694] Loaded metadata for table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]
I20261118 08:54:28.826082  1579 catalog_manager.cc:694] Loaded metadata for table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]
I20261118 08:54:28.828271  1579 tablet_loader.cc:96] loaded metadata for tablet 39e7203496e0458687be52c3fadd15ea (table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30])
I20261118 08:54:28.828600  1579 tablet_loader.cc:96] loaded metadata for tablet 3e292319f0c44e34b49ae6d579df2ecb (table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a])
I20261118 08:54:28.828903  1579 tablet_loader.cc:96] loaded metadata for tablet 4a6c27574dfb45a1b736aad9c209669e (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2])
I20261118 08:54:28.829214  1579 tablet_loader.cc:96] loaded metadata for tablet 543333f1b0df4517a424aa536902aa29 (table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a])
I20261118 08:54:28.830054  1579 tablet_loader.cc:96] loaded metadata for tablet 8285b9829b7d4250a275063f36e1c93a (table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b])
I20261118 08:54:28.830389  1579 tablet_loader.cc:96] loaded metadata for tablet 8b0f9c9d6eef4b66aaab3bcafd810360 (table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7])
I20261118 08:54:28.830726  1579 tablet_loader.cc:96] loaded metadata for tablet a810b27a15594c708bfed8f4a5129912 (table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8])
I20261118 08:54:28.831005  1579 tablet_loader.cc:96] loaded metadata for tablet d546b7f20a7f4d3b9e5612080df04be6 (table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70])
I20261118 08:54:28.831290  1579 tablet_loader.cc:96] loaded metadata for tablet d92e535f2bf44815b6c003b56dd25b79 (table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957])
I20261118 08:54:28.831568  1579 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 08:54:28.832562  1579 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:28.832638  1579 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 08:54:28.833827  1579 catalog_manager.cc:1555] Loading token signing keys...
I20261118 08:54:28.834650  1579 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Loaded TSK: 0
I20261118 08:54:28.835618  1579 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 08:54:28.836447  1579 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 08:54:28.837217  1579 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 08:54:28.852720  1411 master_service.cc:965] ReplaceTablet: received request to replace tablet a810b27a15594c708bfed8f4a5129912 from {username='slave'} at 127.0.0.1:54800
I20261118 08:54:28.852623  1410 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54800:
name: "default.Table_52bba9536ae942a59235d8087d61576b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:28.853251  1405 master_service.cc:965] ReplaceTablet: received request to replace tablet a810b27a15594c708bfed8f4a5129912 from {username='slave'} at 127.0.0.1:54800
I20261118 08:54:28.853641  1406 master_service.cc:965] ReplaceTablet: received request to replace tablet 4a6c27574dfb45a1b736aad9c209669e from {username='slave'} at 127.0.0.1:54800
I20261118 08:54:28.855685  1404 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54800:
table { table_name: "default.Table_567d34233a4b440a952301f19a6bcb10" } modify_external_catalogs: true
I20261118 08:54:28.857137  1408 master_service.cc:965] ReplaceTablet: received request to replace tablet 8285b9829b7d4250a275063f36e1c93a from {username='slave'} at 127.0.0.1:54800
I20261118 08:54:28.856966  1407 master_service.cc:965] ReplaceTablet: received request to replace tablet 8b0f9c9d6eef4b66aaab3bcafd810360 from {username='slave'} at 127.0.0.1:54800
I20261118 08:54:28.857741  1409 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54800:
name: "default.Table_bfd124a61a904e60bc99bee8716f64cc"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:28.857852  1404 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54800:
table { table_name: "default.Table_567d34233a4b440a952301f19a6bcb10" } modify_external_catalogs: true
I20261118 08:54:28.858095  1403 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54800:
table { table_name: "default.Table_d2d73a819dc64fe0badbe6d84293bfff" } modify_external_catalogs: true
I20261118 08:54:28.858073  1400 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54800:
name: "default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:28.859889  1403 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54800:
table { table_name: "default.Table_d2d73a819dc64fe0badbe6d84293bfff" } modify_external_catalogs: true
F20261118 08:54:28.863770  1471 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_bc1f66261ac64a87912e19a32ce0ca44, message={"server":"","servicePrincipal":"","db":"default","table":"table_bc1f66261ac64a87912e19a32ce0ca44","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_bc1f66261ac64a87912e19a32ce0ca44\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_bc1f66261ac64a87912e19a32ce0ca44\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_bc1f66261ac64a87912e19a32ce0ca44\",\"kudu.table_id\":\"4fcf984156584e30aac26283f71adda2\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_8b08d3e4ed824f1680377a6d5e7518a8, message={"server":"","servicePrincipal":"","db":"default","table":"table_8b08d3e4ed824f1680377a6d5e7518a8","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_8b08d3e4ed824f1680377a6d5e7518a8\",\"kudu.table_id\":\"b42eb739941947f8b7d79fddd923eb52\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
    @     0x7f60ec331d5d  google::LogMessage::Fail() at ??:0
    @     0x7f60ec335b23  google::LogMessage::SendToLog() at ??:0
    @     0x7f60ec33175c  google::LogMessage::Flush() at ??:0
    @     0x7f60ec3319a1  google::LogMessage::~LogMessage() at ??:0
I20261118 08:54:28.878122  1406 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 4a6c27574dfb45a1b736aad9c209669e
I20261118 08:54:28.879335  1411 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet a810b27a15594c708bfed8f4a5129912
I20261118 08:54:28.881014  1406 catalog_manager.cc:6936] ReplaceTablet: tablet 4a6c27574dfb45a1b736aad9c209669e deleted and replaced by tablet 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:54:28.881546  1411 catalog_manager.cc:6936] ReplaceTablet: tablet a810b27a15594c708bfed8f4a5129912 deleted and replaced by tablet c02766f2f8d94f479e0ce2dc03567a7d
I20261118 08:54:28.901758  1411 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54800:
table { table_name: "default.Table_294f03c6875c4da0845888273ea80a2a" } modify_external_catalogs: true
I20261118 08:54:28.902024  1411 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54800:
table { table_name: "default.Table_294f03c6875c4da0845888273ea80a2a" } modify_external_catalogs: true
I20261118 08:54:28.902236   918 tablet_service.cc:1547] Processing DeleteTablet for tablet a810b27a15594c708bfed8f4a5129912 with delete_type TABLET_DATA_DELETED (replaced by tablet c02766f2f8d94f479e0ce2dc03567a7d) from {username='slave'} at 127.0.0.1:44738
I20261118 08:54:28.902384   917 tablet_service.cc:1547] Processing DeleteTablet for tablet 4a6c27574dfb45a1b736aad9c209669e with delete_type TABLET_DATA_DELETED (replaced by tablet 8d92c02e29724e80820a7114d8bbdf7a) from {username='slave'} at 127.0.0.1:44738
I20261118 08:54:28.902592  1406 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54800:
name: "default.Table_738b3d8aface40768cac1e7b01d47046"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:28.902884  1405 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54800:
name: "default.Table_3655f04cfdf84b26839c6e76aac6ebbe"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:28.904846  1407 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 8b0f9c9d6eef4b66aaab3bcafd810360
I20261118 08:54:28.905481  1407 catalog_manager.cc:6936] ReplaceTablet: tablet 8b0f9c9d6eef4b66aaab3bcafd810360 deleted and replaced by tablet fc173583a9584741b28f2c0203504a3a
I20261118 08:54:28.905892  1407 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54800:
name: "default.Table_b5863ec6b7e144f58594f23a0634bf0a"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:28.906720   917 tablet_service.cc:1547] Processing DeleteTablet for tablet 8b0f9c9d6eef4b66aaab3bcafd810360 with delete_type TABLET_DATA_DELETED (replaced by tablet fc173583a9584741b28f2c0203504a3a) from {username='slave'} at 127.0.0.1:44738
I20261118 08:54:28.907140  1408 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 8285b9829b7d4250a275063f36e1c93a
I20261118 08:54:28.908082  1408 catalog_manager.cc:6936] ReplaceTablet: tablet 8285b9829b7d4250a275063f36e1c93a deleted and replaced by tablet d7f4643e2be049d7b2e3249a40c6b2ac
I20261118 08:54:28.908615  1408 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54800:
table { table_name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640" } modify_external_catalogs: true
I20261118 08:54:28.908906  1408 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54800:
table { table_name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640" } modify_external_catalogs: true
I20261118 08:54:28.910432   917 tablet_service.cc:1547] Processing DeleteTablet for tablet 8285b9829b7d4250a275063f36e1c93a with delete_type TABLET_DATA_DELETED (replaced by tablet d7f4643e2be049d7b2e3249a40c6b2ac) from {username='slave'} at 127.0.0.1:44738
    @     0x7f61017b19a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 08:54:28.914885  1191 tablet_service.cc:1547] Processing DeleteTablet for tablet a810b27a15594c708bfed8f4a5129912 with delete_type TABLET_DATA_DELETED (replaced by tablet c02766f2f8d94f479e0ce2dc03567a7d) from {username='slave'} at 127.0.0.1:59224
I20261118 08:54:28.914834  1190 tablet_service.cc:1547] Processing DeleteTablet for tablet 4a6c27574dfb45a1b736aad9c209669e with delete_type TABLET_DATA_DELETED (replaced by tablet 8d92c02e29724e80820a7114d8bbdf7a) from {username='slave'} at 127.0.0.1:59224
I20261118 08:54:28.915279  1188 tablet_service.cc:1547] Processing DeleteTablet for tablet 8285b9829b7d4250a275063f36e1c93a with delete_type TABLET_DATA_DELETED (replaced by tablet d7f4643e2be049d7b2e3249a40c6b2ac) from {username='slave'} at 127.0.0.1:59224
I20261118 08:54:28.915498  1189 tablet_service.cc:1547] Processing DeleteTablet for tablet 8b0f9c9d6eef4b66aaab3bcafd810360 with delete_type TABLET_DATA_DELETED (replaced by tablet fc173583a9584741b28f2c0203504a3a) from {username='slave'} at 127.0.0.1:59224
I20261118 08:54:28.919181  1191 tablet_service.cc:1500] Processing CreateTablet for tablet c02766f2f8d94f479e0ce2dc03567a7d (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.919646  1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02766f2f8d94f479e0ce2dc03567a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.920046  1189 tablet_service.cc:1500] Processing CreateTablet for tablet 8d92c02e29724e80820a7114d8bbdf7a (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.920574  1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92c02e29724e80820a7114d8bbdf7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.921535   917 tablet_service.cc:1500] Processing CreateTablet for tablet c02766f2f8d94f479e0ce2dc03567a7d (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.921973   917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02766f2f8d94f479e0ce2dc03567a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.923552  1054 tablet_service.cc:1547] Processing DeleteTablet for tablet 4a6c27574dfb45a1b736aad9c209669e with delete_type TABLET_DATA_DELETED (replaced by tablet 8d92c02e29724e80820a7114d8bbdf7a) from {username='slave'} at 127.0.0.1:52358
I20261118 08:54:28.924041  1595 tablet_replica.cc:333] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:28.924393  1595 raft_consensus.cc:2261] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.924611  1056 tablet_service.cc:1547] Processing DeleteTablet for tablet a810b27a15594c708bfed8f4a5129912 with delete_type TABLET_DATA_DELETED (replaced by tablet c02766f2f8d94f479e0ce2dc03567a7d) from {username='slave'} at 127.0.0.1:52358
I20261118 08:54:28.924877  1055 tablet_service.cc:1547] Processing DeleteTablet for tablet 8b0f9c9d6eef4b66aaab3bcafd810360 with delete_type TABLET_DATA_DELETED (replaced by tablet fc173583a9584741b28f2c0203504a3a) from {username='slave'} at 127.0.0.1:52358
I20261118 08:54:28.924711  1595 raft_consensus.cc:2290] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.925153   916 tablet_service.cc:1500] Processing CreateTablet for tablet 8d92c02e29724e80820a7114d8bbdf7a (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.925642   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92c02e29724e80820a7114d8bbdf7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.925664  1595 ts_tablet_manager.cc:1962] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.926753  1055 tablet_service.cc:1547] Processing DeleteTablet for tablet 8285b9829b7d4250a275063f36e1c93a with delete_type TABLET_DATA_DELETED (replaced by tablet d7f4643e2be049d7b2e3249a40c6b2ac) from {username='slave'} at 127.0.0.1:52358
I20261118 08:54:28.927448  1055 tablet_service.cc:1500] Processing CreateTablet for tablet 8d92c02e29724e80820a7114d8bbdf7a (DEFAULT_TABLE table=default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.927893  1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d92c02e29724e80820a7114d8bbdf7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.931851  1056 tablet_service.cc:1500] Processing CreateTablet for tablet c02766f2f8d94f479e0ce2dc03567a7d (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.932330  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02766f2f8d94f479e0ce2dc03567a7d. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f61017b0251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 08:54:28.935575  1492 tablet_bootstrap.cc:492] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:28.937563  1492 tablet_bootstrap.cc:654] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.937731  1599 tablet_bootstrap.cc:492] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:28.939713  1599 tablet_bootstrap.cc:654] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
    @     0x7f61017b7ecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 08:54:28.941167  1595 ts_tablet_manager.cc:1975] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 08:54:28.941335  1595 log.cc:1199] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/a810b27a15594c708bfed8f4a5129912
I20261118 08:54:28.941763  1595 ts_tablet_manager.cc:1996] T a810b27a15594c708bfed8f4a5129912 P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:28.942346  1599 tablet_bootstrap.cc:492] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:28.942489  1599 ts_tablet_manager.cc:1449] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20261118 08:54:28.943113  1396 catalog_manager.cc:5065] TS ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): tablet a810b27a15594c708bfed8f4a5129912 (table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]) successfully deleted
I20261118 08:54:28.943297  1599 raft_consensus.cc:360] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:28.943533  1599 raft_consensus.cc:386] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.943598  1595 tablet_replica.cc:333] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:28.943629  1599 raft_consensus.cc:748] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.943852  1595 raft_consensus.cc:2261] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.944039  1595 raft_consensus.cc:2290] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.943859  1599 consensus_queue.cc:261] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:28.944686  1599 ts_tablet_manager.cc:1480] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 08:54:28.945024  1599 tablet_bootstrap.cc:492] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:28.945751  1598 tablet_replica.cc:333] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:28.945940  1598 raft_consensus.cc:2261] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:28.946322  1598 raft_consensus.cc:2290] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.947116  1598 ts_tablet_manager.cc:1962] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.948472  1599 tablet_bootstrap.cc:654] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.948590  1595 ts_tablet_manager.cc:1962] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.954685  1598 ts_tablet_manager.cc:1975] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.954819  1598 log.cc:1199] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/4a6c27574dfb45a1b736aad9c209669e
I20261118 08:54:28.955248  1598 ts_tablet_manager.cc:1996] T 4a6c27574dfb45a1b736aad9c209669e P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:28.955268  1595 ts_tablet_manager.cc:1975] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.955456  1595 log.cc:1199] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/4a6c27574dfb45a1b736aad9c209669e
I20261118 08:54:28.955902  1595 ts_tablet_manager.cc:1996] T 4a6c27574dfb45a1b736aad9c209669e P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:28.956182  1598 tablet_replica.cc:333] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:28.956259  1395 catalog_manager.cc:5065] TS 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): tablet 4a6c27574dfb45a1b736aad9c209669e (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]) successfully deleted
I20261118 08:54:28.956377  1598 raft_consensus.cc:2261] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.956533  1598 raft_consensus.cc:2290] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.957016  1598 ts_tablet_manager.cc:1962] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.957012  1396 catalog_manager.cc:5065] TS ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): tablet 4a6c27574dfb45a1b736aad9c209669e (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]) successfully deleted
I20261118 08:54:28.957006  1189 tablet_service.cc:1500] Processing CreateTablet for tablet fc173583a9584741b28f2c0203504a3a (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.957404  1595 tablet_replica.cc:333] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:28.957459  1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc173583a9584741b28f2c0203504a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.957588  1595 raft_consensus.cc:2261] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.957661  1054 tablet_service.cc:1500] Processing CreateTablet for tablet fc173583a9584741b28f2c0203504a3a (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.958022  1054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc173583a9584741b28f2c0203504a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.958320   918 tablet_service.cc:1500] Processing CreateTablet for tablet fc173583a9584741b28f2c0203504a3a (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.958752   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc173583a9584741b28f2c0203504a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.961407  1191 tablet_service.cc:1500] Processing CreateTablet for tablet d7f4643e2be049d7b2e3249a40c6b2ac (DEFAULT_TABLE table=default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.961794  1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f4643e2be049d7b2e3249a40c6b2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.963734  1055 tablet_service.cc:1500] Processing CreateTablet for tablet d7f4643e2be049d7b2e3249a40c6b2ac (DEFAULT_TABLE table=default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.964154  1055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f4643e2be049d7b2e3249a40c6b2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.967142  1599 tablet_bootstrap.cc:492] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:28.967324  1599 ts_tablet_manager.cc:1449] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.004s
I20261118 08:54:28.967584  1597 tablet_replica.cc:333] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:28.967784  1597 raft_consensus.cc:2261] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.968007  1597 raft_consensus.cc:2290] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.968329  1599 raft_consensus.cc:360] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:28.968506  1597 ts_tablet_manager.cc:1962] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.968546  1599 raft_consensus.cc:386] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.968627  1599 raft_consensus.cc:748] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.968957  1599 consensus_queue.cc:261] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:28.971697  1599 ts_tablet_manager.cc:1480] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
    @     0x7f61017b7d49  std::_Function_handler<>::_M_invoke() at ??:0
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 08:54:28.973032  1599 tablet_bootstrap.cc:492] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:28.973680   916 tablet_service.cc:1500] Processing CreateTablet for tablet d7f4643e2be049d7b2e3249a40c6b2ac (DEFAULT_TABLE table=default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:28.974110   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f4643e2be049d7b2e3249a40c6b2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:28.957769  1595 raft_consensus.cc:2290] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.975972  1599 tablet_bootstrap.cc:654] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.976070  1595 ts_tablet_manager.cc:1962] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
    @     0x7f60ed77adc9  kudu::Thread::SuperviseThread() at ??:0
    @     0x7f60f0e556db  start_thread at ??:0
I20261118 08:54:28.979650  1604 tablet_bootstrap.cc:492] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:28.981792  1492 tablet_bootstrap.cc:492] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:28.981971  1492 ts_tablet_manager.cc:1449] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.046s	user 0.005s	sys 0.003s
I20261118 08:54:28.982764  1492 raft_consensus.cc:360] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:28.983376  1492 raft_consensus.cc:386] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.983482  1492 raft_consensus.cc:748] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.983771  1492 consensus_queue.cc:261] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:28.984412  1599 tablet_bootstrap.cc:492] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:28.984561  1599 ts_tablet_manager.cc:1449] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.012s	user 0.006s	sys 0.003s
I20261118 08:54:28.985205  1598 ts_tablet_manager.cc:1975] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 08:54:28.985333  1598 log.cc:1199] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/a810b27a15594c708bfed8f4a5129912
I20261118 08:54:28.985391  1599 raft_consensus.cc:360] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:28.985625  1599 raft_consensus.cc:386] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:28.985680  1598 ts_tablet_manager.cc:1996] T a810b27a15594c708bfed8f4a5129912 P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:28.985721  1599 raft_consensus.cc:748] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:28.986002  1599 consensus_queue.cc:261] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:28.986531  1598 tablet_replica.cc:333] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:28.986706  1598 raft_consensus.cc:2261] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:28.986865  1598 raft_consensus.cc:2290] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
    @     0x7f60e98c561f  clone at ??:0
I20261118 08:54:28.988775  1595 ts_tablet_manager.cc:1975] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.988931  1595 log.cc:1199] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/8b0f9c9d6eef4b66aaab3bcafd810360
I20261118 08:54:28.989209  1599 ts_tablet_manager.cc:1480] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.000s
I20261118 08:54:28.989393  1595 ts_tablet_manager.cc:1996] T 8b0f9c9d6eef4b66aaab3bcafd810360 P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:28.990067  1492 ts_tablet_manager.cc:1480] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.000s
I20261118 08:54:28.990407  1595 tablet_replica.cc:333] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:28.990641  1595 raft_consensus.cc:2261] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:28.990988  1595 raft_consensus.cc:2290] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:28.991914  1595 ts_tablet_manager.cc:1962] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.992282  1599 tablet_bootstrap.cc:492] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:28.992913  1598 ts_tablet_manager.cc:1962] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:28.994145  1599 tablet_bootstrap.cc:654] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.994863  1492 tablet_bootstrap.cc:492] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:28.996616  1492 tablet_bootstrap.cc:654] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:28.998629  1595 ts_tablet_manager.cc:1975] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.998812  1595 log.cc:1199] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/8285b9829b7d4250a275063f36e1c93a
I20261118 08:54:28.999225  1595 ts_tablet_manager.cc:1996] T 8285b9829b7d4250a275063f36e1c93a P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:28.999255  1598 ts_tablet_manager.cc:1975] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:28.999365  1598 log.cc:1199] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/8b0f9c9d6eef4b66aaab3bcafd810360
I20261118 08:54:28.999778  1598 ts_tablet_manager.cc:1996] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:29.000658  1598 tablet_replica.cc:333] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:29.000841  1598 raft_consensus.cc:2261] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:29.000993  1598 raft_consensus.cc:2290] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:29.001811  1598 ts_tablet_manager.cc:1962] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:29.002457  1597 ts_tablet_manager.cc:1975] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 08:54:29.002573  1597 log.cc:1199] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/a810b27a15594c708bfed8f4a5129912
I20261118 08:54:29.002689  1604 tablet_bootstrap.cc:654] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:29.003016  1597 ts_tablet_manager.cc:1996] T a810b27a15594c708bfed8f4a5129912 P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:29.004101  1597 tablet_replica.cc:333] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:29.004312  1597 raft_consensus.cc:2261] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:29.004483  1597 raft_consensus.cc:2290] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:29.004942  1492 tablet_bootstrap.cc:492] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:29.005107  1492 ts_tablet_manager.cc:1449] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.010s	user 0.003s	sys 0.000s
I20261118 08:54:29.006004  1492 raft_consensus.cc:360] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:29.006228  1492 raft_consensus.cc:386] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:29.006331  1492 raft_consensus.cc:748] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:29.006661  1492 consensus_queue.cc:261] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:29.008411  1492 ts_tablet_manager.cc:1480] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.003s	user 0.003s	sys 0.000s
I20261118 08:54:29.009193  1492 tablet_bootstrap.cc:492] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:29.012319  1492 tablet_bootstrap.cc:654] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:29.012897  1597 ts_tablet_manager.cc:1962] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:29.014358  1599 tablet_bootstrap.cc:492] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:29.014552  1599 ts_tablet_manager.cc:1449] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.001s
I20261118 08:54:29.014847  1492 tablet_bootstrap.cc:492] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:29.015009  1492 ts_tablet_manager.cc:1449] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.006s	user 0.003s	sys 0.000s
I20261118 08:54:29.015478  1599 raft_consensus.cc:360] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.015691  1599 raft_consensus.cc:386] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:29.015772  1599 raft_consensus.cc:748] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:29.015894  1492 raft_consensus.cc:360] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.016084  1492 raft_consensus.cc:386] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:29.016161  1492 raft_consensus.cc:748] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:29.016094  1599 consensus_queue.cc:261] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.016448  1492 consensus_queue.cc:261] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.016976  1492 ts_tablet_manager.cc:1480] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20261118 08:54:29.017108  1598 ts_tablet_manager.cc:1975] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:29.017258  1598 log.cc:1199] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/8285b9829b7d4250a275063f36e1c93a
I20261118 08:54:29.017373  1492 tablet_bootstrap.cc:492] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:29.017697  1598 ts_tablet_manager.cc:1996] T 8285b9829b7d4250a275063f36e1c93a P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:29.019327  1492 tablet_bootstrap.cc:654] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:29.019726  1599 ts_tablet_manager.cc:1480] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 08:54:29.024278  1604 tablet_bootstrap.cc:492] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:29.024483  1604 ts_tablet_manager.cc:1449] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.045s	user 0.006s	sys 0.007s
I20261118 08:54:29.025434  1604 raft_consensus.cc:360] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:29.025656  1604 raft_consensus.cc:386] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:29.025751  1604 raft_consensus.cc:748] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:29.026031  1604 consensus_queue.cc:261] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:29.026680  1604 ts_tablet_manager.cc:1480] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.001s
I20261118 08:54:29.027009  1604 tablet_bootstrap.cc:492] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:29.029435  1604 tablet_bootstrap.cc:654] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:29.033851  1597 ts_tablet_manager.cc:1975] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:29.033988  1597 log.cc:1199] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/4a6c27574dfb45a1b736aad9c209669e
I20261118 08:54:29.034363  1597 ts_tablet_manager.cc:1996] T 4a6c27574dfb45a1b736aad9c209669e P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:29.036602  1597 tablet_replica.cc:333] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:29.036787  1597 raft_consensus.cc:2261] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:29.036991  1597 raft_consensus.cc:2290] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:29.037220  1604 tablet_bootstrap.cc:492] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:29.037427  1604 ts_tablet_manager.cc:1449] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.010s	user 0.003s	sys 0.006s
I20261118 08:54:29.037804  1597 ts_tablet_manager.cc:1962] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:29.038371  1604 raft_consensus.cc:360] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.038599  1604 raft_consensus.cc:386] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:29.038669  1604 raft_consensus.cc:748] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:29.039005  1604 consensus_queue.cc:261] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.040548  1604 ts_tablet_manager.cc:1480] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.001s
I20261118 08:54:29.041715  1604 tablet_bootstrap.cc:492] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:29.043889  1604 tablet_bootstrap.cc:654] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:29.044816  1492 tablet_bootstrap.cc:492] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:29.044983  1492 ts_tablet_manager.cc:1449] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.009s
I20261118 08:54:29.045784  1604 tablet_bootstrap.cc:492] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:29.045761  1492 raft_consensus.cc:360] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.045914  1604 ts_tablet_manager.cc:1449] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20261118 08:54:29.045966  1492 raft_consensus.cc:386] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:29.046063  1492 raft_consensus.cc:748] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:29.046283  1492 consensus_queue.cc:261] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.047713  1604 raft_consensus.cc:360] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.047925  1604 raft_consensus.cc:386] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:29.048012  1604 raft_consensus.cc:748] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:29.048277  1604 consensus_queue.cc:261] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.050726  1604 ts_tablet_manager.cc:1480] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 08:54:29.050971  1492 ts_tablet_manager.cc:1480] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.001s
I20261118 08:54:29.053212  1345 raft_consensus.cc:499] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:29.053390  1345 raft_consensus.cc:521] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:29.053701  1597 ts_tablet_manager.cc:1975] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:29.053839  1597 log.cc:1199] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/8285b9829b7d4250a275063f36e1c93a
I20261118 08:54:29.054075  1345 leader_election.cc:290] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:29.054282  1597 ts_tablet_manager.cc:1996] T 8285b9829b7d4250a275063f36e1c93a P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:29.054986   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c02766f2f8d94f479e0ce2dc03567a7d" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:29.055193   938 raft_consensus.cc:2490] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 0.
I20261118 08:54:29.055348  1604 tablet_bootstrap.cc:492] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:29.055714  1012 leader_election.cc:304] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:29.056020  1345 raft_consensus.cc:2826] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:29.056099  1345 raft_consensus.cc:499] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:29.056185  1345 raft_consensus.cc:3096] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.056350  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c02766f2f8d94f479e0ce2dc03567a7d" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:29.056560  1211 raft_consensus.cc:2490] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 0.
I20261118 08:54:29.057531  1604 tablet_bootstrap.cc:654] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:29.057936  1597 tablet_replica.cc:333] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:29.058156  1597 raft_consensus.cc:2261] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:29.058753  1597 raft_consensus.cc:2290] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:29.058985  1345 raft_consensus.cc:521] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:29.059505  1597 ts_tablet_manager.cc:1962] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:29.060072  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c02766f2f8d94f479e0ce2dc03567a7d" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:29.060222  1345 leader_election.cc:290] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 election: Requested vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:29.060278  1211 raft_consensus.cc:3096] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.063593  1211 raft_consensus.cc:2490] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
I20261118 08:54:29.063670   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c02766f2f8d94f479e0ce2dc03567a7d" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:29.063874   938 raft_consensus.cc:3096] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.064239  1011 leader_election.cc:304] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68; no voters: 
I20261118 08:54:29.064589  1345 raft_consensus.cc:2826] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:29.066946   938 raft_consensus.cc:2490] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
I20261118 08:54:29.073475  1597 ts_tablet_manager.cc:1975] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:29.073618  1597 log.cc:1199] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/8b0f9c9d6eef4b66aaab3bcafd810360
I20261118 08:54:29.073993  1597 ts_tablet_manager.cc:1996] T 8b0f9c9d6eef4b66aaab3bcafd810360 P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:29.077414  1604 tablet_bootstrap.cc:492] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:29.077587  1604 ts_tablet_manager.cc:1449] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.010s
I20261118 08:54:29.078325  1604 raft_consensus.cc:360] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.078516  1604 raft_consensus.cc:386] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:29.078576  1604 raft_consensus.cc:748] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:29.078835  1604 consensus_queue.cc:261] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.078892  1345 raft_consensus.cc:705] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 1 LEADER]: Becoming Leader. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Running, Role: LEADER
I20261118 08:54:29.079208  1345 consensus_queue.cc:238] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:29.079319  1604 ts_tablet_manager.cc:1480] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 08:54:29.093987  1617 consensus_queue.cc:1049] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:29.103856  1617 consensus_queue.cc:1049] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:29.184353  1502 raft_consensus.cc:499] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:29.184571  1502 raft_consensus.cc:521] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.185498  1502 leader_election.cc:290] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:29.186062   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d7f4643e2be049d7b2e3249a40c6b2ac" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:29.186086  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d7f4643e2be049d7b2e3249a40c6b2ac" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:29.186312   938 raft_consensus.cc:2490] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:29.186321  1075 raft_consensus.cc:2490] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:29.186769  1147 leader_election.cc:304] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:29.187071  1502 raft_consensus.cc:2826] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:29.187189  1502 raft_consensus.cc:499] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:29.187291  1502 raft_consensus.cc:3096] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.190193  1502 raft_consensus.cc:521] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.190884  1502 leader_election.cc:290] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Requested vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:29.191450   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d7f4643e2be049d7b2e3249a40c6b2ac" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:29.191435  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d7f4643e2be049d7b2e3249a40c6b2ac" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:29.191670  1075 raft_consensus.cc:3096] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.191669   938 raft_consensus.cc:3096] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.194696  1075 raft_consensus.cc:2490] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:29.194698   938 raft_consensus.cc:2490] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:29.195295  1147 leader_election.cc:304] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:29.195685  1502 raft_consensus.cc:2826] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:29.196048  1502 raft_consensus.cc:705] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [term 1 LEADER]: Becoming Leader. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Running, Role: LEADER
I20261118 08:54:29.196377  1502 consensus_queue.cc:238] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.200181  1626 consensus_queue.cc:1049] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:29.202478  1502 consensus_queue.cc:1049] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 08:54:29.221094  1145 connection.cc:573] server connection from 127.0.0.1:59224 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:29.221416   871 connection.cc:573] server connection from 127.0.0.1:44738 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:29.221423  1010 connection.cc:573] server connection from 127.0.0.1:52358 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:29.221741  1264 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:29.226984  1567 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:29.225814 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.190:42341 (local address 127.0.72.129:42431)
1118 08:54:29.225973 (+   159us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:29.225989 (+    16us) client_negotiation.cc:175] Beginning negotiation
1118 08:54:29.226103 (+   114us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 08:54:29.226952 (+   849us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":63}
W20261118 08:54:29.227737  1631 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:29.223983 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.190:42341 (local address 127.0.0.1:54816)
1118 08:54:29.224406 (+   423us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:29.224424 (+    18us) client_negotiation.cc:175] Beginning negotiation
1118 08:54:29.224556 (+   132us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 08:54:29.227699 (+  3143us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":751,"threads_started":1}
W20261118 08:54:29.229310  1565 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:29.226188 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.190:42341 (local address 127.0.72.130:58141)
1118 08:54:29.226329 (+   141us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:29.226345 (+    16us) client_negotiation.cc:175] Beginning negotiation
1118 08:54:29.226451 (+   106us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 08:54:29.229283 (+  2832us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":48}
W20261118 08:54:29.229801  1566 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:29.226581 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.190:42341 (local address 127.0.72.131:35307)
1118 08:54:29.226714 (+   133us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:29.226729 (+    15us) client_negotiation.cc:175] Beginning negotiation
1118 08:54:29.226829 (+   100us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 08:54:29.229776 (+  2947us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":44}
I20261118 08:54:29.271831  1510 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:29.272025  1510 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.272631  1510 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:29.273303  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8d92c02e29724e80820a7114d8bbdf7a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:29.273275  1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8d92c02e29724e80820a7114d8bbdf7a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:29.273514  1075 raft_consensus.cc:2490] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:29.273553  1210 raft_consensus.cc:2490] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:29.274156   873 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:29.274734  1510 raft_consensus.cc:2826] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:29.274843  1510 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:29.274933  1510 raft_consensus.cc:3096] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.276991  1510 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.277599  1510 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Requested vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:29.278119  1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8d92c02e29724e80820a7114d8bbdf7a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:29.278316  1210 raft_consensus.cc:3096] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.278267  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8d92c02e29724e80820a7114d8bbdf7a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:29.278494  1075 raft_consensus.cc:3096] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.280494  1210 raft_consensus.cc:2490] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:29.280887   873 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:29.281147  1510 raft_consensus.cc:2826] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:29.281301  1510 raft_consensus.cc:705] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Becoming Leader. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Running, Role: LEADER
I20261118 08:54:29.281626  1075 raft_consensus.cc:2490] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:29.282887  1510 consensus_queue.cc:238] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.287612  1560 consensus_queue.cc:1049] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:29.288010  1510 consensus_queue.cc:1049] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:29.368247  1510 raft_consensus.cc:499] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:29.368485  1510 raft_consensus.cc:521] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.369315  1510 leader_election.cc:290] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:29.369869  1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "fc173583a9584741b28f2c0203504a3a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:29.369941  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "fc173583a9584741b28f2c0203504a3a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:29.370129  1210 raft_consensus.cc:2490] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:29.370223  1075 raft_consensus.cc:2490] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:29.370555   873 leader_election.cc:304] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:29.370820  1510 raft_consensus.cc:2826] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:29.370944  1510 raft_consensus.cc:499] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:29.371047  1510 raft_consensus.cc:3096] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.373263  1510 raft_consensus.cc:521] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.373885  1510 leader_election.cc:290] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Requested vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:29.374677  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "fc173583a9584741b28f2c0203504a3a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:29.374876  1075 raft_consensus.cc:3096] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.375541  1210 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "fc173583a9584741b28f2c0203504a3a" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:29.375802  1210 raft_consensus.cc:3096] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:29.378446  1075 raft_consensus.cc:2490] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:29.378859   873 leader_election.cc:304] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:29.379009  1210 raft_consensus.cc:2490] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:29.379173  1510 raft_consensus.cc:2826] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:29.379305  1510 raft_consensus.cc:705] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Becoming Leader. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Running, Role: LEADER
I20261118 08:54:29.379662  1510 consensus_queue.cc:238] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:29.382709  1558 consensus_queue.cc:1049] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:29.384151  1558 consensus_queue.cc:1049] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:29.455065  1641 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:29.455246  1641 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:29.455938  1641 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:29.457999   657 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:29.458168   658 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:29.458282   658 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:29.458511  1641 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 08:54:29.542229   290 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task3eSafg/build/asan/bin/kudu pid 639
I20261118 08:54:29.543421   703 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 08:54:29.675859   290 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task3eSafg/build/asan/bin/kudu with pid 639
I20261118 08:54:30.336423   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--webserver_interface=127.0.72.189
--webserver_port=35929
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 08:54:30.680246  1653 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:30.680591  1653 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:30.680701  1653 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:30.689247  1653 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:30.689358  1653 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:30.689433  1653 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:30.689497  1653 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:30.699949  1653 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=35929
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:30.701694  1653 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:30.703778  1653 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:30.711786  1661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:30.711869  1662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:30.712631  1664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:30.713330  1653 server_base.cc:1061] running on GCE node
I20261118 08:54:30.714040  1653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:30.715507  1653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:30.716702  1653 hybrid_clock.cc:648] HybridClock initialized: now 1794992070716649 us; error 44 us; skew 500 ppm
I20261118 08:54:30.717003  1653 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:30.719137  1653 webserver.cc:577] Webserver started at http://127.0.72.189:35929/ using document root <none> and password file <none>
I20261118 08:54:30.719911  1653 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:30.720008  1653 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:30.726109  1653 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.002s
I20261118 08:54:30.729140  1670 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:30.730661  1653 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20261118 08:54:30.730842  1653 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
uuid: "857a577b88ba46cc846ecd1a25a722ca"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:30.731626  1653 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:30.753530  1653 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:30.754446  1653 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:30.754731  1653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:30.774755  1653 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:33939
I20261118 08:54:30.774757  1721 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:33939 every 8 connection(s)
I20261118 08:54:30.776564  1653 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
I20261118 08:54:30.784112   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 1653
I20261118 08:54:30.784190  1722 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:30.788595  1722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap starting.
I20261118 08:54:30.800938  1722 log.cc:826] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:30.835799  1722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=28 ignored=0} mutations{seen=29 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 08:54:30.836321  1722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap complete.
I20261118 08:54:30.841758  1722 raft_consensus.cc:360] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:30.842485  1722 raft_consensus.cc:748] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857a577b88ba46cc846ecd1a25a722ca, State: Initialized, Role: FOLLOWER
I20261118 08:54:30.843135  1722 consensus_queue.cc:261] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 3.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:30.843972  1727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:30.844249  1727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:30.845108  1722 sys_catalog.cc:565] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:30.856894  1741 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:30.857098  1741 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: loading cluster ID for follower catalog manager: success
I20261118 08:54:30.861529  1741 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: acquiring CA information for follower catalog manager: success
I20261118 08:54:30.864830  1741 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:30.866236  1653 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:54:30.870672  1653 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:30.921033  1258 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:30.921142  1123 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:30.925102   985 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:30.925268  1687 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:35731; Asking this server to re-register.
I20261118 08:54:30.925395  1686 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:42731; Asking this server to re-register.
I20261118 08:54:30.926259  1123 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:30.926622  1687 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:48349; Asking this server to re-register.
I20261118 08:54:30.927291   985 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:30.927832  1258 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:30.928539  1687 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:30.929653  1686 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:30.929838  1685 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:31.289635  1727 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:31.289908  1727 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:31.291962  1727 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:31.292603  1672 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:31.294041  1672 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:31.294247  1671 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:31.294368  1671 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:31.294775  1727 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 08:54:31.902907  1754 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:31.903082  1754 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:31.903862  1754 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:31.906141  1672 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:31.906347  1671 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:31.906461  1671 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:31.906694  1754 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 08:54:32.628974  1770 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:32.629154  1770 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:32.629761  1770 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:32.631855  1671 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:32.632030  1672 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:32.632192  1672 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:32.632450  1770 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 08:54:32.972749   290 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task3eSafg/build/asan/bin/kudu pid 1653
I20261118 08:54:32.979182  1717 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 08:54:33.084019   290 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task3eSafg/build/asan/bin/kudu with pid 1653
I20261118 08:54:33.605788   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--webserver_interface=127.0.72.189
--webserver_port=35929
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 08:54:33.948292  1790 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:33.948655  1790 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:33.948773  1790 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:33.957535  1790 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:33.957633  1790 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:33.957687  1790 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:33.957742  1790 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:33.968124  1790 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=35929
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:33.969779  1790 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:33.971678  1790 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:33.980048  1799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:33.980067  1800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:33.980266  1802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:33.981252  1790 server_base.cc:1061] running on GCE node
I20261118 08:54:33.982055  1790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:33.983558  1790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:33.984758  1790 hybrid_clock.cc:648] HybridClock initialized: now 1794992073984701 us; error 50 us; skew 500 ppm
I20261118 08:54:33.985090  1790 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:33.987232  1790 webserver.cc:577] Webserver started at http://127.0.72.189:35929/ using document root <none> and password file <none>
I20261118 08:54:33.988013  1790 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:33.988123  1790 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:33.993777  1790 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20261118 08:54:33.996807  1808 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:33.998427  1790 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20261118 08:54:33.998595  1790 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
uuid: "857a577b88ba46cc846ecd1a25a722ca"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:33.999312  1790 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:34.014524  1790 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:34.015448  1790 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:34.015741  1790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:34.034358  1790 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:33939
I20261118 08:54:34.034348  1859 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:33939 every 8 connection(s)
I20261118 08:54:34.036114  1790 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
I20261118 08:54:34.042789   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 1790
I20261118 08:54:34.043184  1860 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:34.047022  1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap starting.
I20261118 08:54:34.059885  1860 log.cc:826] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:34.091569  1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=28 ignored=0} mutations{seen=29 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 08:54:34.092113  1860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap complete.
I20261118 08:54:34.098269  1860 raft_consensus.cc:360] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:34.099370  1860 raft_consensus.cc:748] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857a577b88ba46cc846ecd1a25a722ca, State: Initialized, Role: FOLLOWER
I20261118 08:54:34.100063  1860 consensus_queue.cc:261] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 3.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:34.100914  1864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:34.101200  1864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:34.102216  1860 sys_catalog.cc:565] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:34.117811  1878 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:34.112875  1790 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:54:34.118331  1790 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:34.117982  1878 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: loading cluster ID for follower catalog manager: success
I20261118 08:54:34.122696  1878 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: acquiring CA information for follower catalog manager: success
I20261118 08:54:34.125782  1878 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:34.434473   985 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:34.437698  1825 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:35571; Asking this server to re-register.
I20261118 08:54:34.438459   985 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:34.440364  1825 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:34.450368  1123 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:34.451653  1258 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:34.452072  1825 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:34163; Asking this server to re-register.
I20261118 08:54:34.452872  1123 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:34.452898  1825 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:50553; Asking this server to re-register.
I20261118 08:54:34.453461  1258 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:34.454926  1825 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:34.455084  1824 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:34.703913  1887 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:34.704208  1887 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:34.706130  1887 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:34.707976  1809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:34.709331  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:34.709542  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:34.709676  1809 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:34.710012  1887 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 08:54:35.271229  1894 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:35.271445  1894 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:35.272192  1894 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:35.274032  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:35.274194  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:35.274315  1810 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:35.274534  1894 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 08:54:35.878988  1904 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:35.879158  1904 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:35.879787  1904 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:35.885272  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:35.885445  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:35.885576  1810 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:35.885815  1904 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 08:54:36.671113  1920 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:36.671316  1920 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:36.672034  1920 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:36.673853  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:36.674011  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:36.674132  1809 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:36.674335  1920 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 08:54:37.498605   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--webserver_interface=127.0.72.190
--webserver_port=41231
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 08:54:37.840474  1936 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:37.840798  1936 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:37.840876  1936 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:37.849210  1936 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:37.849303  1936 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:37.849356  1936 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:37.849403  1936 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:37.860102  1936 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=41231
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:37.861647  1936 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:37.863332  1936 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:37.871068  1941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:37.871091  1942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:37.871731  1944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:37.872495  1936 server_base.cc:1061] running on GCE node
I20261118 08:54:37.873270  1936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:37.874717  1936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:37.875929  1936 hybrid_clock.cc:648] HybridClock initialized: now 1794992077875876 us; error 33 us; skew 500 ppm
I20261118 08:54:37.876283  1936 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:37.878607  1936 webserver.cc:577] Webserver started at http://127.0.72.190:41231/ using document root <none> and password file <none>
I20261118 08:54:37.879313  1936 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:37.879446  1936 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:37.885092  1936 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.004s	sys 0.000s
I20261118 08:54:37.888119  1950 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:37.889607  1936 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20261118 08:54:37.889775  1936 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
uuid: "78d4026bc230421ea10b8c68a9ef7871"
format_stamp: "Formatted at 2026-11-18 08:54:21 on dist-test-slave-m8c8"
I20261118 08:54:37.890481  1936 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:37.909832  1936 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:37.910746  1936 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:37.911073  1936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:37.932791  1936 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42341
I20261118 08:54:37.932793  2005 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42341 every 8 connection(s)
I20261118 08:54:37.934542  1936 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
I20261118 08:54:37.936503   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 1936
I20261118 08:54:37.943734  2006 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:37.949684  2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap starting.
I20261118 08:54:37.965703  2006 log.cc:826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:37.968868   984 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:38.003865  2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap replayed 1/1 log segments. Stats: ops{read=40 overwritten=0 applied=40 ignored=0} inserts{seen=28 ignored=0} mutations{seen=29 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 08:54:38.004470  2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap complete.
I20261118 08:54:38.010102  2006 raft_consensus.cc:360] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:38.010878  2006 raft_consensus.cc:748] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Initialized, Role: FOLLOWER
I20261118 08:54:38.011572  2006 consensus_queue.cc:261] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 40, Last appended: 3.40, Last appended by leader: 40, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:38.012372  2013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:38.012611  2013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:38.013458  2006 sys_catalog.cc:565] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:38.028909  2027 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:38.029009  2027 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: loading cluster ID for follower catalog manager: success
I20261118 08:54:38.033143  2027 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: acquiring CA information for follower catalog manager: success
I20261118 08:54:38.036168  2027 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:38.048614  1936 master_runner.cc:187] Error getting master registration for 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111), 
I20261118 08:54:38.052482  1936 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:38.431475  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:38.434144  1967 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:36541; Asking this server to re-register.
I20261118 08:54:38.434937  1122 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:38.437098  1967 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:38.438434  1257 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:38.440115  1967 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:47025; Asking this server to re-register.
I20261118 08:54:38.440724  1257 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:38.441984  1967 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:38.474819  1967 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:60377; Asking this server to re-register.
I20261118 08:54:38.475494   984 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:38.476765  1967 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:38.547330  2037 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:38.547628  2037 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:38.549396  2037 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:38.550100  1835 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 4 candidate_status { last_received { term: 3 index: 40 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca" is_pre_election: true
W20261118 08:54:38.550227  1951 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:38.550536  1835 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 3.
I20261118 08:54:38.551163  1952 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 
I20261118 08:54:38.551549  2037 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20261118 08:54:38.551712  2037 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20261118 08:54:38.551739  1951 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:38.551821  2037 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 3 FOLLOWER]: Advancing to term 4
I20261118 08:54:38.555420  2037 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:38.556025  2037 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 4 election: Requested vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:38.556504  1835 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 4 candidate_status { last_received { term: 3 index: 40 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca"
I20261118 08:54:38.556701  1835 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 3 FOLLOWER]: Advancing to term 4
W20261118 08:54:38.557291  1951 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:38.560171  1835 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 4.
I20261118 08:54:38.560583  1952 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1
I20261118 08:54:38.560910  2037 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Leader election won for term 4
I20261118 08:54:38.561303  2037 raft_consensus.cc:705] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 LEADER]: Becoming Leader. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Running, Role: LEADER
I20261118 08:54:38.561751  2037 consensus_queue.cc:238] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 40, Committed index: 40, Last appended: 3.40, Last appended by leader: 40, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:38.564471  2040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 4 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:38.564848  2040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
W20261118 08:54:38.566103  1951 consensus_peers.cc:597] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 -> Peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Couldn't send request to peer 6c7d9f52503f41e4bafe2821e6fd6af1. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:38.566125  1835 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Refusing update from remote peer 78d4026bc230421ea10b8c68a9ef7871: Log matching property violated. Preceding OpId in replica: term: 3 index: 40. Preceding OpId from leader: term: 4 index: 41. (index mismatch)
I20261118 08:54:38.566212  2044 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 08:54:38.566967  2040 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 41, Last known committed idx: 40, Time since last communication: 0.000s
I20261118 08:54:38.569252  2044 catalog_manager.cc:694] Loaded metadata for table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30]
I20261118 08:54:38.569825  2044 catalog_manager.cc:694] Loaded metadata for table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a]
I20261118 08:54:38.570199  2044 catalog_manager.cc:694] Loaded metadata for table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70]
I20261118 08:54:38.570542  2044 catalog_manager.cc:694] Loaded metadata for table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]
I20261118 08:54:38.570642  2045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 4 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:38.570847  2044 catalog_manager.cc:694] Loaded metadata for table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]
I20261118 08:54:38.570859  2045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:38.571182  2044 catalog_manager.cc:694] Loaded metadata for table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]
I20261118 08:54:38.571520  2044 catalog_manager.cc:694] Loaded metadata for table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]
I20261118 08:54:38.571931  2044 catalog_manager.cc:694] Loaded metadata for table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]
I20261118 08:54:38.572305  2044 catalog_manager.cc:694] Loaded metadata for table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]
I20261118 08:54:38.573462  2042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:38.573683  2042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:38.574127  2045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:38.574325  2045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:38.575042  2044 tablet_loader.cc:96] loaded metadata for tablet 39e7203496e0458687be52c3fadd15ea (table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30])
I20261118 08:54:38.575328  2044 tablet_loader.cc:96] loaded metadata for tablet 3e292319f0c44e34b49ae6d579df2ecb (table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a])
I20261118 08:54:38.575809  2044 tablet_loader.cc:96] loaded metadata for tablet 543333f1b0df4517a424aa536902aa29 (table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a])
I20261118 08:54:38.576488  2044 tablet_loader.cc:96] loaded metadata for tablet 8d92c02e29724e80820a7114d8bbdf7a (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2])
I20261118 08:54:38.576895  2044 tablet_loader.cc:96] loaded metadata for tablet c02766f2f8d94f479e0ce2dc03567a7d (table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8])
I20261118 08:54:38.577153  2044 tablet_loader.cc:96] loaded metadata for tablet d546b7f20a7f4d3b9e5612080df04be6 (table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70])
I20261118 08:54:38.577440  2044 tablet_loader.cc:96] loaded metadata for tablet d7f4643e2be049d7b2e3249a40c6b2ac (table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b])
I20261118 08:54:38.577713  2044 tablet_loader.cc:96] loaded metadata for tablet d92e535f2bf44815b6c003b56dd25b79 (table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957])
I20261118 08:54:38.577973  2044 tablet_loader.cc:96] loaded metadata for tablet fc173583a9584741b28f2c0203504a3a (table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7])
I20261118 08:54:38.578210  2044 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 08:54:38.579172  2044 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:38.579246  2044 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 08:54:38.580497  2044 catalog_manager.cc:1555] Loading token signing keys...
I20261118 08:54:38.581327  2044 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Loaded TSK: 0
I20261118 08:54:38.582141  2044 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 08:54:38.582975  2044 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 08:54:38.583799  2044 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 08:54:38.656008  1967 master_service.cc:965] ReplaceTablet: received request to replace tablet d7f4643e2be049d7b2e3249a40c6b2ac from {username='slave'} at 127.0.0.1:43154
I20261118 08:54:38.656168  1965 master_service.cc:965] ReplaceTablet: received request to replace tablet 8d92c02e29724e80820a7114d8bbdf7a from {username='slave'} at 127.0.0.1:43154
I20261118 08:54:38.656546  1963 master_service.cc:965] ReplaceTablet: received request to replace tablet 39e7203496e0458687be52c3fadd15ea from {username='slave'} at 127.0.0.1:43154
I20261118 08:54:38.657352  1960 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43154:
table { table_name: "default.Table_41d62c0cf6b54938b8e1770b859e56e8" } new_table_name: "default.Table_eefa15fa899c4b1e9162efce0db7e4af" modify_external_catalogs: true
I20261118 08:54:38.657423  1961 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43154:
table { table_name: "default.Table_5f1fb0758cf1413eaed1d8b4717aa902" } new_table_name: "default.Table_d9954c97f0054586ab7cf9e5e98b3fa4" modify_external_catalogs: true
I20261118 08:54:38.657898  1959 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43154:
table { table_name: "default.Table_bc1f66261ac64a87912e19a32ce0ca44" } new_table_name: "default.Table_7fe6a97ebd294ce68601eaf9f252d0ee" modify_external_catalogs: true
I20261118 08:54:38.658116  1964 master_service.cc:965] ReplaceTablet: received request to replace tablet 8d92c02e29724e80820a7114d8bbdf7a from {username='slave'} at 127.0.0.1:43154
I20261118 08:54:38.656033  1966 master_service.cc:965] ReplaceTablet: received request to replace tablet d92e535f2bf44815b6c003b56dd25b79 from {username='slave'} at 127.0.0.1:43154
I20261118 08:54:38.658931  1958 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43154:
name: "default.Table_0327d29a2f19450b872c54c554d7654e"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:38.660022  1962 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43154:
name: "default.Table_8f6feb81c1374d42bc1e0acb856b75cd"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
F20261118 08:54:38.667511  2026 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_bc1f66261ac64a87912e19a32ce0ca44, message={"server":"","servicePrincipal":"","db":"default","table":"table_bc1f66261ac64a87912e19a32ce0ca44","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_bc1f66261ac64a87912e19a32ce0ca44\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_bc1f66261ac64a87912e19a32ce0ca44\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_bc1f66261ac64a87912e19a32ce0ca44\",\"kudu.table_id\":\"4fcf984156584e30aac26283f71adda2\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_8b08d3e4ed824f1680377a6d5e7518a8, message={"server":"","servicePrincipal":"","db":"default","table":"table_8b08d3e4ed824f1680377a6d5e7518a8","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_8b08d3e4ed824f1680377a6d5e7518a8\",\"kudu.table_id\":\"b42eb739941947f8b7d79fddd923eb52\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20261118 08:54:38.669618  1966 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet d92e535f2bf44815b6c003b56dd25b79
I20261118 08:54:38.669919  1967 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet d7f4643e2be049d7b2e3249a40c6b2ac
    @     0x7fe33e914d5d  google::LogMessage::Fail() at ??:0
I20261118 08:54:38.671886  1966 catalog_manager.cc:6936] ReplaceTablet: tablet d92e535f2bf44815b6c003b56dd25b79 deleted and replaced by tablet 86bf01af75674f5491b024ac43a063b0
I20261118 08:54:38.672379  1966 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43154:
name: "default.Table_82d1541f550a4d5482cff56810c97a22"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
    @     0x7fe33e918b23  google::LogMessage::SendToLog() at ??:0
I20261118 08:54:38.674481  1967 catalog_manager.cc:6936] ReplaceTablet: tablet d7f4643e2be049d7b2e3249a40c6b2ac deleted and replaced by tablet 748947f292334e7a97abfa599634006d
I20261118 08:54:38.674811  1967 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43154:
table { table_name: "default.Table_1ee6cacf31104df284b253928039759b" } new_table_name: "default.Table_29f622723bbb4a47970de8920bf7d580" modify_external_catalogs: true
    @     0x7fe33e91475c  google::LogMessage::Flush() at ??:0
    @     0x7fe33e9149a1  google::LogMessage::~LogMessage() at ??:0
    @     0x7fe353d949a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
    @     0x7fe353d93251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
    @     0x7fe353d9aecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 08:54:38.700053  1965 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:54:38.700685  1965 catalog_manager.cc:6936] ReplaceTablet: tablet 8d92c02e29724e80820a7114d8bbdf7a deleted and replaced by tablet 3c8c4245c7ac4d8ca2e7fa521b6d49d3
I20261118 08:54:38.701074  1965 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43154:
name: "default.Table_903cefd8f2a74c89be4bf41b84d26ba4"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:38.701952  1964 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43154:
name: "default.Table_52bba9536ae942a59235d8087d61576b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:38.702536  1963 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 39e7203496e0458687be52c3fadd15ea
I20261118 08:54:38.703162  1963 catalog_manager.cc:6936] ReplaceTablet: tablet 39e7203496e0458687be52c3fadd15ea deleted and replaced by tablet 50744c2d59124a22ac0a76f2f8e60974
    @     0x7fe353d9ad49  std::_Function_handler<>::_M_invoke() at ??:0
I20261118 08:54:38.703526  1963 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43154:
name: "default.Table_bfd124a61a904e60bc99bee8716f64cc"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
    @     0x7fe33fd5ddc9  kudu::Thread::SuperviseThread() at ??:0
    @     0x7fe3434386db  start_thread at ??:0
    @     0x7fe33bea861f  clone at ??:0
I20261118 08:54:38.712870   916 tablet_service.cc:1547] Processing DeleteTablet for tablet d92e535f2bf44815b6c003b56dd25b79 with delete_type TABLET_DATA_DELETED (replaced by tablet 86bf01af75674f5491b024ac43a063b0) from {username='slave'} at 127.0.0.1:37254
I20261118 08:54:38.712965   917 tablet_service.cc:1547] Processing DeleteTablet for tablet d7f4643e2be049d7b2e3249a40c6b2ac with delete_type TABLET_DATA_DELETED (replaced by tablet 748947f292334e7a97abfa599634006d) from {username='slave'} at 127.0.0.1:37254
I20261118 08:54:38.713096   918 tablet_service.cc:1547] Processing DeleteTablet for tablet 8d92c02e29724e80820a7114d8bbdf7a with delete_type TABLET_DATA_DELETED (replaced by tablet 3c8c4245c7ac4d8ca2e7fa521b6d49d3) from {username='slave'} at 127.0.0.1:37254
I20261118 08:54:38.713243   915 tablet_service.cc:1547] Processing DeleteTablet for tablet 39e7203496e0458687be52c3fadd15ea with delete_type TABLET_DATA_DELETED (replaced by tablet 50744c2d59124a22ac0a76f2f8e60974) from {username='slave'} at 127.0.0.1:37254
I20261118 08:54:38.715421  2061 tablet_replica.cc:333] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:38.715626  2061 raft_consensus.cc:2261] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:38.715602  1054 tablet_service.cc:1547] Processing DeleteTablet for tablet d7f4643e2be049d7b2e3249a40c6b2ac with delete_type TABLET_DATA_DELETED (replaced by tablet 748947f292334e7a97abfa599634006d) from {username='slave'} at 127.0.0.1:42700
I20261118 08:54:38.715698  1055 tablet_service.cc:1547] Processing DeleteTablet for tablet d92e535f2bf44815b6c003b56dd25b79 with delete_type TABLET_DATA_DELETED (replaced by tablet 86bf01af75674f5491b024ac43a063b0) from {username='slave'} at 127.0.0.1:42700
I20261118 08:54:38.715850  2061 raft_consensus.cc:2290] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:38.716259  2062 tablet_replica.cc:333] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:38.716382  1054 tablet_service.cc:1547] Processing DeleteTablet for tablet 8d92c02e29724e80820a7114d8bbdf7a with delete_type TABLET_DATA_DELETED (replaced by tablet 3c8c4245c7ac4d8ca2e7fa521b6d49d3) from {username='slave'} at 127.0.0.1:42700
I20261118 08:54:38.716439  2061 ts_tablet_manager.cc:1962] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:38.716457  2062 raft_consensus.cc:2261] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:38.716691  2062 raft_consensus.cc:2290] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:38.716753  1054 tablet_service.cc:1547] Processing DeleteTablet for tablet 39e7203496e0458687be52c3fadd15ea with delete_type TABLET_DATA_DELETED (replaced by tablet 50744c2d59124a22ac0a76f2f8e60974) from {username='slave'} at 127.0.0.1:42700
I20261118 08:54:38.717181  2062 ts_tablet_manager.cc:1962] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:38.721899  2062 ts_tablet_manager.cc:1975] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:38.722026  2062 log.cc:1199] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/d7f4643e2be049d7b2e3249a40c6b2ac
I20261118 08:54:38.722479  2062 ts_tablet_manager.cc:1996] T d7f4643e2be049d7b2e3249a40c6b2ac P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:38.723043  2061 ts_tablet_manager.cc:1975] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:38.723160  2061 log.cc:1199] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/d7f4643e2be049d7b2e3249a40c6b2ac
I20261118 08:54:38.723651  2061 ts_tablet_manager.cc:1996] T d7f4643e2be049d7b2e3249a40c6b2ac P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:38.723691  2062 tablet_replica.cc:333] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:38.723891  2062 raft_consensus.cc:2261] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:38.724050  2062 raft_consensus.cc:2290] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:38.724591  2062 ts_tablet_manager.cc:1962] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:38.724761  2061 tablet_replica.cc:333] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:38.724941  2061 raft_consensus.cc:2261] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:38.725240  2061 raft_consensus.cc:2290] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:38.725767  2061 ts_tablet_manager.cc:1962] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:38.731061  2062 ts_tablet_manager.cc:1975] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:38.731192  2062 log.cc:1199] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/d92e535f2bf44815b6c003b56dd25b79
I20261118 08:54:38.731587  2062 ts_tablet_manager.cc:1996] T d92e535f2bf44815b6c003b56dd25b79 P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:38.732056  2061 ts_tablet_manager.cc:1975] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:38.732203  2061 log.cc:1199] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/d92e535f2bf44815b6c003b56dd25b79
I20261118 08:54:38.732549  2061 ts_tablet_manager.cc:1996] T d92e535f2bf44815b6c003b56dd25b79 P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:38.732853  2062 tablet_replica.cc:333] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:38.733029  2062 raft_consensus.cc:2261] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:38.733203  2062 raft_consensus.cc:2290] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:38.733735  2061 tablet_replica.cc:333] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:38.733758  2062 ts_tablet_manager.cc:1962] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:38.733911  2061 raft_consensus.cc:2261] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:38.734229  2061 raft_consensus.cc:2290] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:38.734766  2061 ts_tablet_manager.cc:1962] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:38.740310  2062 ts_tablet_manager.cc:1975] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:38.740425  2062 log.cc:1199] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:54:38.740833  2062 ts_tablet_manager.cc:1996] T 8d92c02e29724e80820a7114d8bbdf7a P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:38.741091  2061 ts_tablet_manager.cc:1975] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:38.741205  2061 log.cc:1199] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:54:38.741598  2061 ts_tablet_manager.cc:1996] T 8d92c02e29724e80820a7114d8bbdf7a P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:38.742005  2062 tablet_replica.cc:333] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:38.742197  2062 raft_consensus.cc:2261] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:38.742363  2062 raft_consensus.cc:2290] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:38.742674  2061 tablet_replica.cc:333] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:38.742874  2061 raft_consensus.cc:2261] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:38.742931  2062 ts_tablet_manager.cc:1962] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:38.743027  2061 raft_consensus.cc:2290] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:38.743607  2061 ts_tablet_manager.cc:1962] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:38.749228  2062 ts_tablet_manager.cc:1975] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:38.749365  2062 log.cc:1199] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/39e7203496e0458687be52c3fadd15ea
I20261118 08:54:38.749732  2062 ts_tablet_manager.cc:1996] T 39e7203496e0458687be52c3fadd15ea P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:38.749951  2061 ts_tablet_manager.cc:1975] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:38.750082  2061 log.cc:1199] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/39e7203496e0458687be52c3fadd15ea
I20261118 08:54:38.750525  2061 ts_tablet_manager.cc:1996] T 39e7203496e0458687be52c3fadd15ea P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
W20261118 08:54:38.883275  1012 connection.cc:573] server connection from 127.0.0.1:42700 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:38.883280   873 connection.cc:573] server connection from 127.0.0.1:37254 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:38.883319  1809 connection.cc:573] server connection from 127.0.0.1:41760 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:38.883894  1264 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:38.884480  1264 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.190:42341)
W20261118 08:54:38.884816  1264 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:38.884991  1264 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:38.885178  1264 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:38.927958  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 1: this message will repeat every 5th retry.
W20261118 08:54:39.090116  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:39.119756  1790 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:54:39.119920  1790 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:39.211485  2076 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:39.211715  2076 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:39.212397  2076 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:39.214368  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:39.214526  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:39.214643  1809 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:39.214879  2076 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20261118 08:54:39.309316  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 1: this message will repeat every 5th retry.
W20261118 08:54:39.456259  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:39.764822  2079 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:39.765025  2079 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:39.765702  2079 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:39.766674  1810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20261118 08:54:39.767710  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:39.767871  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:39.767995  1810 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:39.768234  2079 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20261118 08:54:40.776459   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--webserver_interface=127.0.72.190
--webserver_port=41231
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 08:54:40.799005  2092 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:40.799183  2092 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:40.799950  2092 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:40.802079  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:40.802532  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:40.802661  1809 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:40.802950  2092 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20261118 08:54:41.126612  2091 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:41.126947  2091 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:41.127067  2091 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:41.136092  2091 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:41.136204  2091 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:41.136276  2091 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:41.136336  2091 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:41.146843  2091 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=41231
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:41.148597  2091 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:41.150419  2091 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:41.158424  2103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:41.158455  2102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:41.159643  2105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:41.159700  2091 server_base.cc:1061] running on GCE node
I20261118 08:54:41.160454  2091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:41.161831  2091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:41.163043  2091 hybrid_clock.cc:648] HybridClock initialized: now 1794992081163002 us; error 30 us; skew 500 ppm
I20261118 08:54:41.163507  2091 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:41.165699  2091 webserver.cc:577] Webserver started at http://127.0.72.190:41231/ using document root <none> and password file <none>
I20261118 08:54:41.166421  2091 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:41.166548  2091 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:41.172041  2091 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.000s
I20261118 08:54:41.174877  2111 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:41.176355  2091 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20261118 08:54:41.176560  2091 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
uuid: "78d4026bc230421ea10b8c68a9ef7871"
format_stamp: "Formatted at 2026-11-18 08:54:21 on dist-test-slave-m8c8"
I20261118 08:54:41.177266  2091 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:41.205526  2091 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:41.206455  2091 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:41.206732  2091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:41.225314  2091 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42341
I20261118 08:54:41.225338  2162 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42341 every 8 connection(s)
I20261118 08:54:41.227650  2091 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
I20261118 08:54:41.233944   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 2091
I20261118 08:54:41.236660  2163 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:41.240811  2163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap starting.
I20261118 08:54:41.246886  1257 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:41.256824  2163 log.cc:826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:41.291499  2163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap replayed 1/1 log segments. Stats: ops{read=46 overwritten=0 applied=45 ignored=0} inserts{seen=32 ignored=0} mutations{seen=33 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20261118 08:54:41.292163  2163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap complete.
I20261118 08:54:41.297416  2163 raft_consensus.cc:360] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:41.299978  2163 raft_consensus.cc:748] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Initialized, Role: FOLLOWER
I20261118 08:54:41.300644  2163 consensus_queue.cc:261] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 45, Last appended: 4.46, Last appended by leader: 46, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:41.301347  2170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:41.301606  2170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:41.302492  2163 sys_catalog.cc:565] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:41.322429  2184 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:41.322548  2184 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: loading cluster ID for follower catalog manager: success
I20261118 08:54:41.333088  2184 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: acquiring CA information for follower catalog manager: success
I20261118 08:54:41.338531  2184 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:41.344064  2085 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:41.344270  2085 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:41.345038  2085 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:54:41.346235  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:54:41.346608  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:54:41.346745  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:41.347000  2085 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:54:41.347980  2091 master_runner.cc:187] Error getting master registration for 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111), 
I20261118 08:54:41.353579  2091 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:41.451195  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:41.453548  2128 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:35739; Asking this server to re-register.
I20261118 08:54:41.454272  1122 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:41.456065  2128 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:41.490257   984 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:41.491521  2128 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:36255; Asking this server to re-register.
I20261118 08:54:41.492142   984 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:41.493414  2128 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:41.619026  2085 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:41.619220  2085 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:41.619940  2085 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:54:41.620870  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:54:41.621011  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:54:41.621135  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:41.621389  2085 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:54:41.752985  2127 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:38145; Asking this server to re-register.
I20261118 08:54:41.753643  1257 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:41.754921  2127 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:41.785933  2170 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:41.786216  2170 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:41.788117  2170 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:41.788689  1835 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 5 candidate_status { last_received { term: 4 index: 46 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca" is_pre_election: true
W20261118 08:54:41.788769  2112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:41.788897  1835 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 4.
I20261118 08:54:41.789543  2113 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 
I20261118 08:54:41.789920  2170 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20261118 08:54:41.790056  2170 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:41.790176  2170 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 4 FOLLOWER]: Advancing to term 5
W20261118 08:54:41.790189  2112 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:41.793629  2170 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:41.794229  2170 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 5 election: Requested vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:41.794701  1835 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 5 candidate_status { last_received { term: 4 index: 46 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca"
I20261118 08:54:41.794909  1835 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 4 FOLLOWER]: Advancing to term 5
W20261118 08:54:41.795624  2112 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:41.797029  1835 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 5.
I20261118 08:54:41.797428  2113 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1
I20261118 08:54:41.797689  2170 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Leader election won for term 5
I20261118 08:54:41.798125  2170 raft_consensus.cc:705] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 LEADER]: Becoming Leader. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Running, Role: LEADER
I20261118 08:54:41.798553  2170 consensus_queue.cc:238] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 45, Committed index: 45, Last appended: 4.46, Last appended by leader: 46, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:41.801198  2190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 5 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:41.801499  2190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:41.802472  1835 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Refusing update from remote peer 78d4026bc230421ea10b8c68a9ef7871: Log matching property violated. Preceding OpId in replica: term: 4 index: 46. Preceding OpId from leader: term: 5 index: 47. (index mismatch)
W20261118 08:54:41.803197  2112 consensus_peers.cc:597] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 -> Peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Couldn't send request to peer 6c7d9f52503f41e4bafe2821e6fd6af1. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:41.803460  2190 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 47, Last known committed idx: 45, Time since last communication: 0.000s
I20261118 08:54:41.805589  2195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 5 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:41.805917  2195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:41.816071  2190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:41.816321  2190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:41.817168  2195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:41.817390  2195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:41.818313  2194 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 08:54:41.820760  2194 catalog_manager.cc:694] Loaded metadata for table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30]
I20261118 08:54:41.821252  2194 catalog_manager.cc:694] Loaded metadata for table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a]
I20261118 08:54:41.821573  2194 catalog_manager.cc:694] Loaded metadata for table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70]
I20261118 08:54:41.821861  2194 catalog_manager.cc:694] Loaded metadata for table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]
I20261118 08:54:41.822126  2194 catalog_manager.cc:694] Loaded metadata for table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]
I20261118 08:54:41.822499  2194 catalog_manager.cc:694] Loaded metadata for table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]
I20261118 08:54:41.822829  2194 catalog_manager.cc:694] Loaded metadata for table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]
I20261118 08:54:41.823119  2194 catalog_manager.cc:694] Loaded metadata for table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]
I20261118 08:54:41.823454  2194 catalog_manager.cc:694] Loaded metadata for table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]
I20261118 08:54:41.825683  2194 tablet_loader.cc:96] loaded metadata for tablet 3c8c4245c7ac4d8ca2e7fa521b6d49d3 (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2])
I20261118 08:54:41.826037  2194 tablet_loader.cc:96] loaded metadata for tablet 3e292319f0c44e34b49ae6d579df2ecb (table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a])
I20261118 08:54:41.826409  2194 tablet_loader.cc:96] loaded metadata for tablet 50744c2d59124a22ac0a76f2f8e60974 (table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30])
I20261118 08:54:41.826645  2194 tablet_loader.cc:96] loaded metadata for tablet 543333f1b0df4517a424aa536902aa29 (table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a])
I20261118 08:54:41.827014  2194 tablet_loader.cc:96] loaded metadata for tablet 748947f292334e7a97abfa599634006d (table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b])
I20261118 08:54:41.827411  2194 tablet_loader.cc:96] loaded metadata for tablet 86bf01af75674f5491b024ac43a063b0 (table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957])
I20261118 08:54:41.828136  2194 tablet_loader.cc:96] loaded metadata for tablet c02766f2f8d94f479e0ce2dc03567a7d (table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8])
I20261118 08:54:41.828378  2194 tablet_loader.cc:96] loaded metadata for tablet d546b7f20a7f4d3b9e5612080df04be6 (table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70])
I20261118 08:54:41.829104  2194 tablet_loader.cc:96] loaded metadata for tablet fc173583a9584741b28f2c0203504a3a (table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7])
I20261118 08:54:41.829449  2194 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 08:54:41.830688  2194 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:41.830760  2194 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 08:54:41.832032  2194 catalog_manager.cc:1555] Loading token signing keys...
I20261118 08:54:41.832919  2194 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Loaded TSK: 0
I20261118 08:54:41.833741  2194 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 08:54:41.834811  2194 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 08:54:41.835723  2194 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 08:54:41.909436  2127 master_service.cc:965] ReplaceTablet: received request to replace tablet fc173583a9584741b28f2c0203504a3a from {username='slave'} at 127.0.0.1:43814
I20261118 08:54:41.909548  2128 master_service.cc:965] ReplaceTablet: received request to replace tablet 3e292319f0c44e34b49ae6d579df2ecb from {username='slave'} at 127.0.0.1:43814
I20261118 08:54:41.910156  2126 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43814:
table { table_name: "default.Table_ef26d2c593994e118ff3d01a0f5a9a1d" } new_table_name: "default.Table_0d932fab32a1414582d1f8370b6e2b23" modify_external_catalogs: true
I20261118 08:54:41.912067  2119 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43814:
table { table_name: "default.Table_d2d73a819dc64fe0badbe6d84293bfff" } modify_external_catalogs: true
I20261118 08:54:41.914013  2119 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43814:
table { table_name: "default.Table_d2d73a819dc64fe0badbe6d84293bfff" } modify_external_catalogs: true
I20261118 08:54:41.910920  2121 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43814:
name: "default.Table_3655f04cfdf84b26839c6e76aac6ebbe"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:41.913192  2120 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43814:
name: "default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:41.911432  2122 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43814:
name: "default.Table_738b3d8aface40768cac1e7b01d47046"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:41.913676  2123 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43814:
table { table_name: "default.Table_294f03c6875c4da0845888273ea80a2a" } modify_external_catalogs: true
I20261118 08:54:41.913538  2125 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43814:
table { table_name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640" } modify_external_catalogs: true
I20261118 08:54:41.913583  2124 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43814:
name: "default.Table_b5863ec6b7e144f58594f23a0634bf0a"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:41.915833  2123 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43814:
table { table_name: "default.Table_294f03c6875c4da0845888273ea80a2a" } modify_external_catalogs: true
I20261118 08:54:41.916203  2125 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43814:
table { table_name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640" } modify_external_catalogs: true
F20261118 08:54:41.920807  2183 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_bc1f66261ac64a87912e19a32ce0ca44, message={"server":"","servicePrincipal":"","db":"default","table":"table_bc1f66261ac64a87912e19a32ce0ca44","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_bc1f66261ac64a87912e19a32ce0ca44\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_bc1f66261ac64a87912e19a32ce0ca44\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_bc1f66261ac64a87912e19a32ce0ca44\",\"kudu.table_id\":\"4fcf984156584e30aac26283f71adda2\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_8b08d3e4ed824f1680377a6d5e7518a8, message={"server":"","servicePrincipal":"","db":"default","table":"table_8b08d3e4ed824f1680377a6d5e7518a8","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_8b08d3e4ed824f1680377a6d5e7518a8\",\"kudu.table_id\":\"b42eb739941947f8b7d79fddd923eb52\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
    @     0x7f99eefd1d5d  google::LogMessage::Fail() at ??:0
    @     0x7f99eefd5b23  google::LogMessage::SendToLog() at ??:0
    @     0x7f99eefd175c  google::LogMessage::Flush() at ??:0
I20261118 08:54:41.926334  2128 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 3e292319f0c44e34b49ae6d579df2ecb
    @     0x7f99eefd19a1  google::LogMessage::~LogMessage() at ??:0
I20261118 08:54:41.931813  2128 catalog_manager.cc:6936] ReplaceTablet: tablet 3e292319f0c44e34b49ae6d579df2ecb deleted and replaced by tablet 368ffd5a7d724f47abeea8f223415eac
I20261118 08:54:41.932560  2128 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43814:
name: "default.Table_8f6feb81c1374d42bc1e0acb856b75cd"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
    @     0x7f9a044519a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 08:54:41.937314   917 tablet_service.cc:1547] Processing DeleteTablet for tablet 3e292319f0c44e34b49ae6d579df2ecb with delete_type TABLET_DATA_DELETED (replaced by tablet 368ffd5a7d724f47abeea8f223415eac) from {username='slave'} at 127.0.0.1:37256
    @     0x7f9a04450251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 08:54:41.944357  2127 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet fc173583a9584741b28f2c0203504a3a
I20261118 08:54:41.944939  2127 catalog_manager.cc:6936] ReplaceTablet: tablet fc173583a9584741b28f2c0203504a3a deleted and replaced by tablet d2ce64bc39634b5882b1beaa68e66cbe
I20261118 08:54:41.945310  2127 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43814:
table { table_name: "default.Table_5f1fb0758cf1413eaed1d8b4717aa902" } new_table_name: "default.Table_d9954c97f0054586ab7cf9e5e98b3fa4" modify_external_catalogs: true
    @     0x7f9a04457ecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 08:54:41.945468   915 tablet_service.cc:1547] Processing DeleteTablet for tablet fc173583a9584741b28f2c0203504a3a with delete_type TABLET_DATA_DELETED (replaced by tablet d2ce64bc39634b5882b1beaa68e66cbe) from {username='slave'} at 127.0.0.1:37256
    @     0x7f9a04457d49  std::_Function_handler<>::_M_invoke() at ??:0
I20261118 08:54:41.950743  2210 tablet_replica.cc:333] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:41.950951  2210 raft_consensus.cc:2261] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Raft consensus shutting down.
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 08:54:41.951332  2210 raft_consensus.cc:2290] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:41.952090  2210 ts_tablet_manager.cc:1962] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
    @     0x7f99f041adc9  kudu::Thread::SuperviseThread() at ??:0
    @     0x7f99f3af56db  start_thread at ??:0
    @     0x7f99ec56561f  clone at ??:0
I20261118 08:54:41.962895  2210 ts_tablet_manager.cc:1975] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:41.963006  2210 log.cc:1199] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/3e292319f0c44e34b49ae6d579df2ecb
I20261118 08:54:41.963457  2210 ts_tablet_manager.cc:1996] T 3e292319f0c44e34b49ae6d579df2ecb P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:41.964551  2210 tablet_replica.cc:333] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:41.964756  2210 raft_consensus.cc:2261] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:41.965077  2210 raft_consensus.cc:2290] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:41.965689  2210 ts_tablet_manager.cc:1962] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:41.970599  2210 ts_tablet_manager.cc:1975] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:41.970714  2210 log.cc:1199] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/fc173583a9584741b28f2c0203504a3a
I20261118 08:54:41.970952  2210 ts_tablet_manager.cc:1996] T fc173583a9584741b28f2c0203504a3a P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
W20261118 08:54:42.124670   873 connection.cc:573] server connection from 127.0.0.1:37256 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:42.124761  1809 connection.cc:573] server connection from 127.0.0.1:41764 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:42.125068   872 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:42.125095  1010 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:42.127063  1264 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.190:42341)
W20261118 08:54:42.127530  1264 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:42.127703  1264 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:42.127985  1264 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.190:42341)
I20261118 08:54:42.459189  2225 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:42.459452  2225 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:42.460191  2225 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:42.462214  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:42.462415  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:42.462533  1809 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:42.462795  2225 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 08:54:43.102944  2235 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:43.103153  2235 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:43.103906  2235 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:43.105993  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:43.106249  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:43.106382  1810 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:43.106643  2235 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 08:54:43.905901  2244 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:43.906155  2244 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:43.906924  2244 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:43.908864  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:43.909019  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:43.909200  1809 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:43.909490  2244 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 08:54:44.121034  1790 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:54:44.121227  1790 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:44.131490  2204 raft_consensus.cc:499] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:44.131685  2204 raft_consensus.cc:521] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:44.132512  2204 leader_election.cc:290] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:44.133042  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3e292319f0c44e34b49ae6d579df2ecb" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
W20261118 08:54:44.133451  1012 leader_election.cc:343] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb
I20261118 08:54:44.133659  1012 leader_election.cc:304] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:44.133914  2204 raft_consensus.cc:2771] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:54:44.262116  2241 raft_consensus.cc:499] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:44.262312  2241 raft_consensus.cc:521] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:44.263151  2241 leader_election.cc:290] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:44.263796  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "fc173583a9584741b28f2c0203504a3a" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
W20261118 08:54:44.264132  1147 leader_election.cc:343] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a
I20261118 08:54:44.264405  1146 leader_election.cc:304] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:44.264662  2241 raft_consensus.cc:2771] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:54:44.395051  2241 raft_consensus.cc:499] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:44.395237  2241 raft_consensus.cc:521] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:44.395877  2241 leader_election.cc:290] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:44.396471  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3e292319f0c44e34b49ae6d579df2ecb" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
W20261118 08:54:44.396667  1147 leader_election.cc:343] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb
I20261118 08:54:44.396729  1075 raft_consensus.cc:2490] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:44.397127  1146 leader_election.cc:304] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68; no voters: ad2789ff5b54489c84966cff78be4721
I20261118 08:54:44.397373  2241 raft_consensus.cc:2826] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261118 08:54:44.397498  2241 raft_consensus.cc:499] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:44.397601  2241 raft_consensus.cc:3096] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Advancing to term 2
I20261118 08:54:44.399966  2241 raft_consensus.cc:521] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:44.400686  2241 leader_election.cc:290] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 election: Requested vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:44.401234  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "3e292319f0c44e34b49ae6d579df2ecb" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:44.401465  1075 raft_consensus.cc:3096] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Advancing to term 2
W20261118 08:54:44.402962  1147 leader_election.cc:343] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb
I20261118 08:54:44.404084  1075 raft_consensus.cc:2490] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 2.
I20261118 08:54:44.404493  1146 leader_election.cc:304] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68; no voters: ad2789ff5b54489c84966cff78be4721
I20261118 08:54:44.404811  2250 raft_consensus.cc:2826] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 2 FOLLOWER]: Leader election won for term 2
I20261118 08:54:44.404959  2250 raft_consensus.cc:705] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [term 2 LEADER]: Becoming Leader. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Running, Role: LEADER
I20261118 08:54:44.405443  2250 consensus_queue.cc:238] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } }
I20261118 08:54:44.408420  1075 raft_consensus.cc:1285] T 3e292319f0c44e34b49ae6d579df2ecb P 213b5f7a64e74b109a3a9387e5975042 [term 2 FOLLOWER]: Refusing update from remote peer 3934fbc351ee4a99add35742e90dfa68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20261118 08:54:44.408514  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:44.409495  2242 consensus_queue.cc:1049] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261118 08:54:44.423717  2204 raft_consensus.cc:499] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:44.423907  2204 raft_consensus.cc:521] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:44.424486  2204 leader_election.cc:290] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:44.424980  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "fc173583a9584741b28f2c0203504a3a" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:44.425171  1211 raft_consensus.cc:2490] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 1.
W20261118 08:54:44.425257  1012 leader_election.cc:343] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a
I20261118 08:54:44.425526  1011 leader_election.cc:304] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68; no voters: ad2789ff5b54489c84966cff78be4721
I20261118 08:54:44.425869  2204 raft_consensus.cc:2826] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261118 08:54:44.426002  2204 raft_consensus.cc:499] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:44.426101  2204 raft_consensus.cc:3096] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Advancing to term 2
I20261118 08:54:44.428121  2204 raft_consensus.cc:521] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:44.428637  2204 leader_election.cc:290] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 election: Requested vote from peers 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:44.429155  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "fc173583a9584741b28f2c0203504a3a" candidate_uuid: "213b5f7a64e74b109a3a9387e5975042" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:44.429349  1211 raft_consensus.cc:3096] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Advancing to term 2
W20261118 08:54:44.429451  1012 leader_election.cc:343] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a
I20261118 08:54:44.431300  1211 raft_consensus.cc:2490] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 213b5f7a64e74b109a3a9387e5975042 in term 2.
I20261118 08:54:44.431685  1011 leader_election.cc:304] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68; no voters: ad2789ff5b54489c84966cff78be4721
I20261118 08:54:44.431959  2204 raft_consensus.cc:2826] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 2 FOLLOWER]: Leader election won for term 2
I20261118 08:54:44.432335  2204 raft_consensus.cc:705] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [term 2 LEADER]: Becoming Leader. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Running, Role: LEADER
I20261118 08:54:44.432679  2204 consensus_queue.cc:238] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:44.434875  1211 raft_consensus.cc:1285] T fc173583a9584741b28f2c0203504a3a P 3934fbc351ee4a99add35742e90dfa68 [term 2 FOLLOWER]: Refusing update from remote peer 213b5f7a64e74b109a3a9387e5975042: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20261118 08:54:44.435186  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:44.435516  2256 consensus_queue.cc:1049] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261118 08:54:44.438470   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--webserver_interface=127.0.72.190
--webserver_port=41231
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 08:54:44.570493  2261 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:44.570724  2261 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:44.571563  2261 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:44.573730  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:44.573909  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:44.574051  1810 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:44.574301  2261 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20261118 08:54:44.775671  2260 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:44.775975  2260 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:44.776087  2260 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:44.785437  2260 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:44.785568  2260 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:44.785653  2260 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:44.785713  2260 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:44.797622  2260 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=41231
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:44.799216  2260 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:44.801024  2260 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:44.810016  2269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:44.809978  2270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:44.810720  2272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:44.811125  2260 server_base.cc:1061] running on GCE node
I20261118 08:54:44.812016  2260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:44.813534  2260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:44.814772  2260 hybrid_clock.cc:648] HybridClock initialized: now 1794992084814718 us; error 40 us; skew 500 ppm
I20261118 08:54:44.815150  2260 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:44.817447  2260 webserver.cc:577] Webserver started at http://127.0.72.190:41231/ using document root <none> and password file <none>
I20261118 08:54:44.818176  2260 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:44.818279  2260 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:44.824108  2260 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.001s
I20261118 08:54:44.827350  2278 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:44.828881  2260 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20261118 08:54:44.829056  2260 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
uuid: "78d4026bc230421ea10b8c68a9ef7871"
format_stamp: "Formatted at 2026-11-18 08:54:21 on dist-test-slave-m8c8"
I20261118 08:54:44.829788  2260 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:44.849324  2260 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:44.850241  2260 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:44.850518  2260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:44.869256  2260 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42341
I20261118 08:54:44.869275  2329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42341 every 8 connection(s)
I20261118 08:54:44.870968  2260 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
I20261118 08:54:44.877192   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 2260
I20261118 08:54:44.878539  2330 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:44.882647  2330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap starting.
I20261118 08:54:44.896541  2330 log.cc:826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:44.938407  2330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap replayed 1/1 log segments. Stats: ops{read=50 overwritten=0 applied=49 ignored=0} inserts{seen=34 ignored=0} mutations{seen=37 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20261118 08:54:44.938997  2330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap complete.
I20261118 08:54:44.946107  2330 raft_consensus.cc:360] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:44.948501  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:44.949528  2330 raft_consensus.cc:748] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Initialized, Role: FOLLOWER
I20261118 08:54:44.950356  2330 consensus_queue.cc:261] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 49, Last appended: 5.50, Last appended by leader: 50, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:44.951246  2336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:44.951610  2336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:44.952493  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:44.952653  2330 sys_catalog.cc:565] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:44.966886  2350 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:44.967099  2350 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: loading cluster ID for follower catalog manager: success
I20261118 08:54:44.971984  2350 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: acquiring CA information for follower catalog manager: success
I20261118 08:54:44.975409  2350 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:44.996165  2260 master_runner.cc:187] Error getting master registration for 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111), 
I20261118 08:54:45.001827  2260 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:45.133347  2250 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:45.133574  2250 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:45.134302  2250 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:54:45.135304  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:54:45.135563  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:54:45.135708  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:45.135991  2250 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:54:45.138609   984 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:45.140461  2295 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:49285; Asking this server to re-register.
I20261118 08:54:45.141134   984 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:45.142781  2295 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:45.171938  2250 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:45.172159  2250 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.172801  2250 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:54:45.173719  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:54:45.173854  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:54:45.174016  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:45.174291  2250 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:54:45.266217  1257 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:45.268138  2295 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:49553; Asking this server to re-register.
I20261118 08:54:45.268877  1257 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:45.270509  2295 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:45.456005  2295 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:54539; Asking this server to re-register.
I20261118 08:54:45.456629  1122 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:45.457814  2295 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:45.562240  2357 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:45.562467  2357 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:45.564275  2357 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:45.564788  1835 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 6 candidate_status { last_received { term: 5 index: 50 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca" is_pre_election: true
I20261118 08:54:45.564999  1835 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 5.
I20261118 08:54:45.565582  2280 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 
W20261118 08:54:45.565613  2279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:45.565954  2357 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20261118 08:54:45.566066  2357 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:45.566169  2357 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 5 FOLLOWER]: Advancing to term 6
W20261118 08:54:45.567098  2279 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:45.569499  2357 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:45.570129  2357 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 6 election: Requested vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:45.570554  1835 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 6 candidate_status { last_received { term: 5 index: 50 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca"
I20261118 08:54:45.570758  1835 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 5 FOLLOWER]: Advancing to term 6
W20261118 08:54:45.571403  2279 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:45.572911  1835 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 6.
I20261118 08:54:45.573312  2280 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1
I20261118 08:54:45.573601  2357 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Leader election won for term 6
I20261118 08:54:45.573964  2357 raft_consensus.cc:705] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 LEADER]: Becoming Leader. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Running, Role: LEADER
I20261118 08:54:45.574389  2357 consensus_queue.cc:238] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 49, Committed index: 49, Last appended: 5.50, Last appended by leader: 50, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:45.576934  2360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 6 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:45.577244  2360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:45.579406  1835 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Refusing update from remote peer 78d4026bc230421ea10b8c68a9ef7871: Log matching property violated. Preceding OpId in replica: term: 5 index: 50. Preceding OpId from leader: term: 6 index: 51. (index mismatch)
W20261118 08:54:45.579671  2279 consensus_peers.cc:597] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 -> Peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Couldn't send request to peer 6c7d9f52503f41e4bafe2821e6fd6af1. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:45.580118  2357 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 51, Last known committed idx: 50, Time since last communication: 0.000s
I20261118 08:54:45.581964  2366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 6 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:45.582159  2366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:45.585362  2366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:45.585556  2366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:45.586051  2357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:45.586223  2357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:45.586616  2364 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 08:54:45.589028  2364 catalog_manager.cc:694] Loaded metadata for table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30]
I20261118 08:54:45.589435  2364 catalog_manager.cc:694] Loaded metadata for table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a]
I20261118 08:54:45.589745  2364 catalog_manager.cc:694] Loaded metadata for table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70]
I20261118 08:54:45.590025  2364 catalog_manager.cc:694] Loaded metadata for table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]
I20261118 08:54:45.590315  2364 catalog_manager.cc:694] Loaded metadata for table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]
I20261118 08:54:45.590612  2364 catalog_manager.cc:694] Loaded metadata for table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]
I20261118 08:54:45.590884  2364 catalog_manager.cc:694] Loaded metadata for table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]
I20261118 08:54:45.591161  2364 catalog_manager.cc:694] Loaded metadata for table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]
I20261118 08:54:45.591490  2364 catalog_manager.cc:694] Loaded metadata for table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]
I20261118 08:54:45.593555  2364 tablet_loader.cc:96] loaded metadata for tablet 368ffd5a7d724f47abeea8f223415eac (table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a])
I20261118 08:54:45.594010  2364 tablet_loader.cc:96] loaded metadata for tablet 3c8c4245c7ac4d8ca2e7fa521b6d49d3 (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2])
I20261118 08:54:45.594566  2364 tablet_loader.cc:96] loaded metadata for tablet 50744c2d59124a22ac0a76f2f8e60974 (table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30])
I20261118 08:54:45.594828  2364 tablet_loader.cc:96] loaded metadata for tablet 543333f1b0df4517a424aa536902aa29 (table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a])
I20261118 08:54:45.595269  2364 tablet_loader.cc:96] loaded metadata for tablet 748947f292334e7a97abfa599634006d (table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b])
I20261118 08:54:45.595700  2364 tablet_loader.cc:96] loaded metadata for tablet 86bf01af75674f5491b024ac43a063b0 (table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957])
I20261118 08:54:45.596383  2364 tablet_loader.cc:96] loaded metadata for tablet c02766f2f8d94f479e0ce2dc03567a7d (table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8])
I20261118 08:54:45.596617  2364 tablet_loader.cc:96] loaded metadata for tablet d2ce64bc39634b5882b1beaa68e66cbe (table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7])
I20261118 08:54:45.596876  2364 tablet_loader.cc:96] loaded metadata for tablet d546b7f20a7f4d3b9e5612080df04be6 (table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70])
I20261118 08:54:45.597560  2364 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 08:54:45.598541  2364 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:45.598621  2364 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 08:54:45.599884  2364 catalog_manager.cc:1555] Loading token signing keys...
I20261118 08:54:45.600752  2364 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Loaded TSK: 0
I20261118 08:54:45.601557  2364 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 08:54:45.602424  2364 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 08:54:45.603241  2364 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 08:54:45.647493  2294 master_service.cc:965] ReplaceTablet: received request to replace tablet 3e292319f0c44e34b49ae6d579df2ecb from {username='slave'} at 127.0.0.1:44548
I20261118 08:54:45.647660  2293 master_service.cc:965] ReplaceTablet: received request to replace tablet c02766f2f8d94f479e0ce2dc03567a7d from {username='slave'} at 127.0.0.1:44548
I20261118 08:54:45.647539  2295 master_service.cc:965] ReplaceTablet: received request to replace tablet 543333f1b0df4517a424aa536902aa29 from {username='slave'} at 127.0.0.1:44548
I20261118 08:54:45.648473  2292 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44548:
table { table_name: "default.Table_567d34233a4b440a952301f19a6bcb10" } modify_external_catalogs: true
I20261118 08:54:45.648810  2292 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44548:
table { table_name: "default.Table_567d34233a4b440a952301f19a6bcb10" } modify_external_catalogs: true
I20261118 08:54:45.648780  2291 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44548:
name: "default.Table_bfd124a61a904e60bc99bee8716f64cc"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:45.649781  2294 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44548:
name: "default.Table_52bba9536ae942a59235d8087d61576b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:45.650588  2290 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44548:
name: "default.Table_903cefd8f2a74c89be4bf41b84d26ba4"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:45.651674  2288 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44548:
table { table_name: "default.Table_294f03c6875c4da0845888273ea80a2a" } modify_external_catalogs: true
I20261118 08:54:45.651759  2289 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44548:
table { table_name: "default.Table_1ee6cacf31104df284b253928039759b" } new_table_name: "default.Table_29f622723bbb4a47970de8920bf7d580" modify_external_catalogs: true
I20261118 08:54:45.651922  2288 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44548:
table { table_name: "default.Table_294f03c6875c4da0845888273ea80a2a" } modify_external_catalogs: true
I20261118 08:54:45.652081  2287 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44548:
name: "default.Table_b5863ec6b7e144f58594f23a0634bf0a"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:45.652942  2286 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44548:
table { table_name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640" } modify_external_catalogs: true
I20261118 08:54:45.653156  2286 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44548:
table { table_name: "default.Table_c3f2df1b938946c19c1ca00d8fcbd640" } modify_external_catalogs: true
F20261118 08:54:45.657433  2349 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_bc1f66261ac64a87912e19a32ce0ca44, message={"server":"","servicePrincipal":"","db":"default","table":"table_bc1f66261ac64a87912e19a32ce0ca44","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_bc1f66261ac64a87912e19a32ce0ca44\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_bc1f66261ac64a87912e19a32ce0ca44\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_bc1f66261ac64a87912e19a32ce0ca44\",\"kudu.table_id\":\"4fcf984156584e30aac26283f71adda2\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_8b08d3e4ed824f1680377a6d5e7518a8, message={"server":"","servicePrincipal":"","db":"default","table":"table_8b08d3e4ed824f1680377a6d5e7518a8","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_8b08d3e4ed824f1680377a6d5e7518a8\",\"kudu.table_id\":\"b42eb739941947f8b7d79fddd923eb52\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20261118 08:54:45.660223  2295 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 543333f1b0df4517a424aa536902aa29
    @     0x7f0dbe599d5d  google::LogMessage::Fail() at ??:0
I20261118 08:54:45.661126  2293 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet c02766f2f8d94f479e0ce2dc03567a7d
I20261118 08:54:45.661912  2293 catalog_manager.cc:6936] ReplaceTablet: tablet c02766f2f8d94f479e0ce2dc03567a7d deleted and replaced by tablet 4eecc49bfe014d1d86c84e8a4f16a5bf
I20261118 08:54:45.661980  2295 catalog_manager.cc:6936] ReplaceTablet: tablet 543333f1b0df4517a424aa536902aa29 deleted and replaced by tablet 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:54:45.662235  2295 master_service.cc:965] ReplaceTablet: received request to replace tablet fc173583a9584741b28f2c0203504a3a from {username='slave'} at 127.0.0.1:44548
    @     0x7f0dbe59db23  google::LogMessage::SendToLog() at ??:0
I20261118 08:54:45.662334  2293 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44548:
table { table_name: "default.Table_ef26d2c593994e118ff3d01a0f5a9a1d" } new_table_name: "default.Table_0d932fab32a1414582d1f8370b6e2b23" modify_external_catalogs: true
I20261118 08:54:45.662700  2295 master_service.cc:965] ReplaceTablet: received request to replace tablet 3e292319f0c44e34b49ae6d579df2ecb from {username='slave'} at 127.0.0.1:44548
I20261118 08:54:45.663203  2295 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44548:
name: "default.Table_738b3d8aface40768cac1e7b01d47046"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
    @     0x7f0dbe59975c  google::LogMessage::Flush() at ??:0
    @     0x7f0dbe5999a1  google::LogMessage::~LogMessage() at ??:0
I20261118 08:54:45.679201  1191 tablet_service.cc:1547] Processing DeleteTablet for tablet c02766f2f8d94f479e0ce2dc03567a7d with delete_type TABLET_DATA_DELETED (replaced by tablet 4eecc49bfe014d1d86c84e8a4f16a5bf) from {username='slave'} at 127.0.0.1:55098
I20261118 08:54:45.679340  1189 tablet_service.cc:1547] Processing DeleteTablet for tablet 543333f1b0df4517a424aa536902aa29 with delete_type TABLET_DATA_DELETED (replaced by tablet 48a6e2d96fe442b6a84b4773993b4f54) from {username='slave'} at 127.0.0.1:55098
I20261118 08:54:45.684916  2379 tablet_replica.cc:333] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:45.685117  2379 raft_consensus.cc:2261] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:45.685356  2379 raft_consensus.cc:2290] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:45.685927  2379 ts_tablet_manager.cc:1962] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:45.687873  1189 tablet_service.cc:1500] Processing CreateTablet for tablet d2ce64bc39634b5882b1beaa68e66cbe (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.688336  1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ce64bc39634b5882b1beaa68e66cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:45.692135  1188 tablet_service.cc:1500] Processing CreateTablet for tablet 48a6e2d96fe442b6a84b4773993b4f54 (DEFAULT_TABLE table=default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.692721  1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48a6e2d96fe442b6a84b4773993b4f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:45.696115   915 tablet_service.cc:1547] Processing DeleteTablet for tablet c02766f2f8d94f479e0ce2dc03567a7d with delete_type TABLET_DATA_DELETED (replaced by tablet 4eecc49bfe014d1d86c84e8a4f16a5bf) from {username='slave'} at 127.0.0.1:37264
I20261118 08:54:45.696417   917 tablet_service.cc:1547] Processing DeleteTablet for tablet 543333f1b0df4517a424aa536902aa29 with delete_type TABLET_DATA_DELETED (replaced by tablet 48a6e2d96fe442b6a84b4773993b4f54) from {username='slave'} at 127.0.0.1:37264
I20261118 08:54:45.696815   914 tablet_service.cc:1500] Processing CreateTablet for tablet 48a6e2d96fe442b6a84b4773993b4f54 (DEFAULT_TABLE table=default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.696942   918 tablet_service.cc:1500] Processing CreateTablet for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.697422   914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48a6e2d96fe442b6a84b4773993b4f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:45.697602   916 tablet_service.cc:1500] Processing CreateTablet for tablet d2ce64bc39634b5882b1beaa68e66cbe (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.697970   916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ce64bc39634b5882b1beaa68e66cbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:45.702206  2379 ts_tablet_manager.cc:1975] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:45.702330  2379 log.cc:1199] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/c02766f2f8d94f479e0ce2dc03567a7d
I20261118 08:54:45.702744  2379 ts_tablet_manager.cc:1996] T c02766f2f8d94f479e0ce2dc03567a7d P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:45.703578   918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:45.703809  2379 tablet_replica.cc:333] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: stopping tablet replica
I20261118 08:54:45.704002  2379 raft_consensus.cc:2261] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:45.704208  2379 raft_consensus.cc:2290] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:45.704780  2379 ts_tablet_manager.cc:1962] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:45.709052  2281 catalog_manager.cc:5065] TS 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): tablet c02766f2f8d94f479e0ce2dc03567a7d (table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]) successfully deleted
I20261118 08:54:45.710289  1054 tablet_service.cc:1547] Processing DeleteTablet for tablet 543333f1b0df4517a424aa536902aa29 with delete_type TABLET_DATA_DELETED (replaced by tablet 48a6e2d96fe442b6a84b4773993b4f54) from {username='slave'} at 127.0.0.1:42714
I20261118 08:54:45.711205  1055 tablet_service.cc:1547] Processing DeleteTablet for tablet c02766f2f8d94f479e0ce2dc03567a7d with delete_type TABLET_DATA_DELETED (replaced by tablet 4eecc49bfe014d1d86c84e8a4f16a5bf) from {username='slave'} at 127.0.0.1:42714
I20261118 08:54:45.711453  1053 tablet_service.cc:1500] Processing CreateTablet for tablet d2ce64bc39634b5882b1beaa68e66cbe (DEFAULT_TABLE table=default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.711581  1056 tablet_service.cc:1500] Processing CreateTablet for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.711984  1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2ce64bc39634b5882b1beaa68e66cbe. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f0dd3a199a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 08:54:45.712085  1052 tablet_service.cc:1500] Processing CreateTablet for tablet 48a6e2d96fe442b6a84b4773993b4f54 (DEFAULT_TABLE table=default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.712599  1052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48a6e2d96fe442b6a84b4773993b4f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:45.715615  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f0dd3a18251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 08:54:45.717371  2379 ts_tablet_manager.cc:1975] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:45.717506  2379 log.cc:1199] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-2/wal/wals/543333f1b0df4517a424aa536902aa29
I20261118 08:54:45.717850  2379 ts_tablet_manager.cc:1996] T 543333f1b0df4517a424aa536902aa29 P 3934fbc351ee4a99add35742e90dfa68: Deleting consensus metadata
I20261118 08:54:45.719834  2281 catalog_manager.cc:5065] TS 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): tablet 543333f1b0df4517a424aa536902aa29 (table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]) successfully deleted
I20261118 08:54:45.723469  2384 tablet_bootstrap.cc:492] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:45.725718  2384 tablet_bootstrap.cc:654] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
    @     0x7f0dd3a1fecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
    @     0x7f0dd3a1fd49  std::_Function_handler<>::_M_invoke() at ??:0
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 08:54:45.735297  1191 tablet_service.cc:1500] Processing CreateTablet for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:45.735878  1191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f0dbf9e2dc9  kudu::Thread::SuperviseThread() at ??:0
    @     0x7f0dc30bd6db  start_thread at ??:0
    @     0x7f0dbbb2d61f  clone at ??:0
I20261118 08:54:45.768440  2387 tablet_bootstrap.cc:492] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:45.770736  2387 tablet_bootstrap.cc:654] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:45.771905  2383 tablet_replica.cc:333] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:45.772131  2383 raft_consensus.cc:2261] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:45.772547  2383 raft_consensus.cc:2290] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:45.773263  2383 ts_tablet_manager.cc:1962] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:45.777240  2385 tablet_bootstrap.cc:492] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:45.779354  2385 tablet_bootstrap.cc:654] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:45.780836  2382 tablet_replica.cc:333] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:45.781045  2382 raft_consensus.cc:2261] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:45.781320  2382 raft_consensus.cc:2290] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:45.781953  2382 ts_tablet_manager.cc:1962] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:45.782316  2387 tablet_bootstrap.cc:492] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:45.782378  2384 tablet_bootstrap.cc:492] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:45.782455  2387 ts_tablet_manager.cc:1449] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.014s	user 0.000s	sys 0.004s
I20261118 08:54:45.782536  2384 ts_tablet_manager.cc:1449] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.019s
I20261118 08:54:45.783213  2387 raft_consensus.cc:360] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.783420  2384 raft_consensus.cc:360] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.783624  2384 raft_consensus.cc:386] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.783722  2384 raft_consensus.cc:748] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.783979  2384 consensus_queue.cc:261] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
W20261118 08:54:45.785959  1011 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.72.131:46447, user_credentials={real_user=slave}} blocked reactor thread for 102812us
I20261118 08:54:45.786330  2387 raft_consensus.cc:386] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.786446  2387 raft_consensus.cc:748] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.786679  2387 consensus_queue.cc:261] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
W20261118 08:54:45.787444  1011 consensus_peers.cc:597] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 -> Peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Couldn't send request to peer 3934fbc351ee4a99add35742e90dfa68. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c02766f2f8d94f479e0ce2dc03567a7d. This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:45.788435  2382 ts_tablet_manager.cc:1975] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:45.788546  2382 log.cc:1199] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/c02766f2f8d94f479e0ce2dc03567a7d
I20261118 08:54:45.788952  2382 ts_tablet_manager.cc:1996] T c02766f2f8d94f479e0ce2dc03567a7d P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:45.790069  2383 ts_tablet_manager.cc:1975] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:45.790195  2383 log.cc:1199] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/543333f1b0df4517a424aa536902aa29
I20261118 08:54:45.790598  2383 ts_tablet_manager.cc:1996] T 543333f1b0df4517a424aa536902aa29 P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:45.790830  2385 tablet_bootstrap.cc:492] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:45.790961  2385 ts_tablet_manager.cc:1449] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.014s	user 0.003s	sys 0.000s
I20261118 08:54:45.791806  2385 raft_consensus.cc:360] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.791985  2385 raft_consensus.cc:386] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.792059  2385 raft_consensus.cc:748] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.792346  2385 consensus_queue.cc:261] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.792900  2385 ts_tablet_manager.cc:1480] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.002s
I20261118 08:54:45.793260  2387 ts_tablet_manager.cc:1480] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.002s
I20261118 08:54:45.793540  2387 tablet_bootstrap.cc:492] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:45.795266  2385 tablet_bootstrap.cc:492] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:45.795974  2387 tablet_bootstrap.cc:654] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:45.796962  2385 tablet_bootstrap.cc:654] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:45.797372   984 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
I20261118 08:54:45.797889  2382 tablet_replica.cc:333] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:45.798087  2382 raft_consensus.cc:2261] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:45.798280  2384 ts_tablet_manager.cc:1480] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20261118 08:54:45.798300  2382 raft_consensus.cc:2290] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:45.798563  2384 tablet_bootstrap.cc:492] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:45.798717  2383 tablet_replica.cc:333] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:45.798894  2383 raft_consensus.cc:2261] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:45.798913  2382 ts_tablet_manager.cc:1962] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:45.799203  2383 raft_consensus.cc:2290] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:45.799755  2383 ts_tablet_manager.cc:1962] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:45.800385  2384 tablet_bootstrap.cc:654] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:45.807165  2387 tablet_bootstrap.cc:492] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:45.807340  2387 ts_tablet_manager.cc:1449] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.014s	user 0.000s	sys 0.004s
I20261118 08:54:45.808202  2387 raft_consensus.cc:360] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.808408  2387 raft_consensus.cc:386] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.808467  2387 raft_consensus.cc:748] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.808926  2387 consensus_queue.cc:261] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.809528  2387 ts_tablet_manager.cc:1480] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 08:54:45.809882  2387 tablet_bootstrap.cc:492] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68: Bootstrap starting.
I20261118 08:54:45.810953  2382 ts_tablet_manager.cc:1975] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:45.811091  2382 log.cc:1199] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/543333f1b0df4517a424aa536902aa29
I20261118 08:54:45.811533  2382 ts_tablet_manager.cc:1996] T 543333f1b0df4517a424aa536902aa29 P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:45.811796  2387 tablet_bootstrap.cc:654] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:45.813758  2385 tablet_bootstrap.cc:492] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:45.813923  2385 ts_tablet_manager.cc:1449] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.001s
I20261118 08:54:45.814698  2385 raft_consensus.cc:360] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.814860  2383 ts_tablet_manager.cc:1975] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:45.814904  2385 raft_consensus.cc:386] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.814980  2385 raft_consensus.cc:748] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.814983  2383 log.cc:1199] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/c02766f2f8d94f479e0ce2dc03567a7d
I20261118 08:54:45.815599  2383 ts_tablet_manager.cc:1996] T c02766f2f8d94f479e0ce2dc03567a7d P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:45.816249  2384 tablet_bootstrap.cc:492] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:45.816320  2387 tablet_bootstrap.cc:492] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68: No bootstrap required, opened a new log
I20261118 08:54:45.816408  2384 ts_tablet_manager.cc:1449] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.000s
I20261118 08:54:45.816449  2387 ts_tablet_manager.cc:1449] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68: Time spent bootstrapping tablet: real 0.007s	user 0.003s	sys 0.000s
I20261118 08:54:45.816603  2385 consensus_queue.cc:261] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.817167  2384 raft_consensus.cc:360] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.817354  2384 raft_consensus.cc:386] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.817420  2384 raft_consensus.cc:748] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.817492  2385 ts_tablet_manager.cc:1480] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.001s
I20261118 08:54:45.817646  2384 consensus_queue.cc:261] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.817993  2385 tablet_bootstrap.cc:492] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:45.817994  2387 raft_consensus.cc:360] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.818112  2384 ts_tablet_manager.cc:1480] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 08:54:45.818153  2387 raft_consensus.cc:386] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.818243  2387 raft_consensus.cc:748] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.818406  2384 tablet_bootstrap.cc:492] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: Bootstrap starting.
I20261118 08:54:45.818445  2387 consensus_queue.cc:261] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.818929  2387 ts_tablet_manager.cc:1480] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20261118 08:54:45.819768  2385 tablet_bootstrap.cc:654] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:45.820111  2384 tablet_bootstrap.cc:654] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:45.824848  2384 tablet_bootstrap.cc:492] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: No bootstrap required, opened a new log
I20261118 08:54:45.824980  2384 ts_tablet_manager.cc:1449] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: Time spent bootstrapping tablet: real 0.007s	user 0.003s	sys 0.003s
I20261118 08:54:45.825731  2384 raft_consensus.cc:360] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.825886  2384 raft_consensus.cc:386] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.825963  2384 raft_consensus.cc:748] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.826211  2384 consensus_queue.cc:261] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.826809  2384 ts_tablet_manager.cc:1480] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.001s
I20261118 08:54:45.828231  2385 tablet_bootstrap.cc:492] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:45.828397  2385 ts_tablet_manager.cc:1449] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.000s
I20261118 08:54:45.829315  2385 raft_consensus.cc:360] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.829492  2385 raft_consensus.cc:386] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:45.829578  2385 raft_consensus.cc:748] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:45.829788  2385 consensus_queue.cc:261] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.830346  2385 ts_tablet_manager.cc:1480] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 08:54:45.902840  2241 raft_consensus.cc:499] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:45.903061  2241 raft_consensus.cc:521] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.903960  2241 leader_election.cc:290] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:45.904490  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4eecc49bfe014d1d86c84e8a4f16a5bf" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:45.904538   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4eecc49bfe014d1d86c84e8a4f16a5bf" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:45.904729  1075 raft_consensus.cc:2490] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:45.904806   938 raft_consensus.cc:2490] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:45.905341  1146 leader_election.cc:304] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, 3934fbc351ee4a99add35742e90dfa68; no voters: 
I20261118 08:54:45.905652  2241 raft_consensus.cc:2826] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:45.905769  2241 raft_consensus.cc:499] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:45.905879  2241 raft_consensus.cc:3096] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:45.907871  2241 raft_consensus.cc:521] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.908429  2241 leader_election.cc:290] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Requested vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:45.908981  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4eecc49bfe014d1d86c84e8a4f16a5bf" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:45.909003   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4eecc49bfe014d1d86c84e8a4f16a5bf" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:45.909184   938 raft_consensus.cc:3096] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:45.909202  1075 raft_consensus.cc:3096] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:45.912144   938 raft_consensus.cc:2490] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:45.912132  1075 raft_consensus.cc:2490] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:45.912635  1147 leader_election.cc:304] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:45.912974  2241 raft_consensus.cc:2826] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:45.913124  2241 raft_consensus.cc:705] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [term 1 LEADER]: Becoming Leader. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Running, Role: LEADER
I20261118 08:54:45.913421  2241 consensus_queue.cc:238] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:45.916931  2250 consensus_queue.cc:1049] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:45.917296  2241 consensus_queue.cc:1049] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 08:54:45.969353  1147 connection.cc:573] server connection from 127.0.0.1:55098 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:45.969331   871 connection.cc:573] server connection from 127.0.0.1:37264 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:45.969583  1145 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:45.969689   872 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:45.969868  1010 connection.cc:573] server connection from 127.0.0.1:42714 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:45.971330  1010 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:45.971489  2240 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:45.970600 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.190:42341 (local address 127.0.72.131:60079)
1118 08:54:45.970873 (+   273us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:45.970893 (+    20us) client_negotiation.cc:175] Beginning negotiation
1118 08:54:45.971022 (+   129us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 08:54:45.971440 (+   418us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":158}
W20261118 08:54:45.971405  1264 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:45.972258  1264 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.190:42341)
W20261118 08:54:45.972851  1264 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:45.973102  1264 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:45.973062  2226 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:45.970722 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.0.72.190:42341 (local address 127.0.72.129:36069)
1118 08:54:45.971022 (+   300us) negotiation.cc:107] Waiting for socket to connect
1118 08:54:45.971036 (+    14us) client_negotiation.cc:175] Beginning negotiation
1118 08:54:45.971128 (+    92us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 08:54:45.973015 (+  1887us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":197}
W20261118 08:54:45.973461  1264 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.190:42341)
I20261118 08:54:46.153340  2391 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:46.153543  2391 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:46.154693  2391 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:46.155129  1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "48a6e2d96fe442b6a84b4773993b4f54" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:46.155332  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "48a6e2d96fe442b6a84b4773993b4f54" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68" is_pre_election: true
I20261118 08:54:46.155421  1076 raft_consensus.cc:2490] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:46.155586  1211 raft_consensus.cc:2490] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 0.
I20261118 08:54:46.155867   873 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:46.156147  2391 raft_consensus.cc:2826] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:46.156244  2391 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:46.156328  2391 raft_consensus.cc:3096] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:46.158239  2391 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:46.158764  2391 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Requested vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:46.159190  1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "48a6e2d96fe442b6a84b4773993b4f54" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:46.159279  1211 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "48a6e2d96fe442b6a84b4773993b4f54" candidate_uuid: "ad2789ff5b54489c84966cff78be4721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3934fbc351ee4a99add35742e90dfa68"
I20261118 08:54:46.159404  1076 raft_consensus.cc:3096] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:46.159466  1211 raft_consensus.cc:3096] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:46.161302  1076 raft_consensus.cc:2490] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:46.161321  1211 raft_consensus.cc:2490] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad2789ff5b54489c84966cff78be4721 in term 1.
I20261118 08:54:46.161687   873 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:46.161967  2391 raft_consensus.cc:2826] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:46.162343  2391 raft_consensus.cc:705] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Becoming Leader. State: Replica: ad2789ff5b54489c84966cff78be4721, State: Running, Role: LEADER
I20261118 08:54:46.162667  2391 consensus_queue.cc:238] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:46.165011  2241 raft_consensus.cc:499] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:46.165171  2241 raft_consensus.cc:521] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:46.165762  2241 leader_election.cc:290] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:46.166210   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d2ce64bc39634b5882b1beaa68e66cbe" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721" is_pre_election: true
I20261118 08:54:46.166421   938 raft_consensus.cc:2490] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:46.166862  1147 leader_election.cc:304] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:46.166977  2413 consensus_queue.cc:1049] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:46.166971  1076 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d2ce64bc39634b5882b1beaa68e66cbe" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042" is_pre_election: true
I20261118 08:54:46.167188  1076 raft_consensus.cc:2490] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 0.
I20261118 08:54:46.167236  2241 raft_consensus.cc:2826] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 08:54:46.167337  2241 raft_consensus.cc:499] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:46.167464  2241 raft_consensus.cc:3096] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:46.167888  2413 consensus_queue.cc:1049] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:46.170892  2241 raft_consensus.cc:521] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:46.171948   938 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d2ce64bc39634b5882b1beaa68e66cbe" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2789ff5b54489c84966cff78be4721"
I20261118 08:54:46.172158   938 raft_consensus.cc:3096] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:46.174952   938 raft_consensus.cc:2490] T d2ce64bc39634b5882b1beaa68e66cbe P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:46.175511  1147 leader_election.cc:304] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721; no voters: 
I20261118 08:54:46.175818  2250 raft_consensus.cc:2826] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 08:54:46.175943  2250 raft_consensus.cc:705] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [term 1 LEADER]: Becoming Leader. State: Replica: 3934fbc351ee4a99add35742e90dfa68, State: Running, Role: LEADER
I20261118 08:54:46.175967  1075 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d2ce64bc39634b5882b1beaa68e66cbe" candidate_uuid: "3934fbc351ee4a99add35742e90dfa68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "213b5f7a64e74b109a3a9387e5975042"
I20261118 08:54:46.176162  1075 raft_consensus.cc:3096] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Advancing to term 1
I20261118 08:54:46.176242  2250 consensus_queue.cc:238] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:46.180442  1075 raft_consensus.cc:2490] T d2ce64bc39634b5882b1beaa68e66cbe P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3934fbc351ee4a99add35742e90dfa68 in term 1.
I20261118 08:54:46.180924  2241 leader_election.cc:290] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 1 election: Requested vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:46.194816  2421 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:46.195036  2421 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:46.197327  2241 consensus_queue.cc:1049] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:46.197964  2241 consensus_queue.cc:1049] T d2ce64bc39634b5882b1beaa68e66cbe P 3934fbc351ee4a99add35742e90dfa68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 08:54:46.204421  2421 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:46.206439  1809 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20261118 08:54:46.223146  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:46.224149  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:46.224355  1810 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:46.224874  2421 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 08:54:46.882912  2436 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:46.883109  2436 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:46.885046  2436 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:46.887039  1809 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:46.887231  1810 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:46.887413  1810 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:46.887705  2436 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 08:54:47.057238   290 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task3eSafg/build/asan/bin/kudu pid 1790
I20261118 08:54:47.063370  1855 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 08:54:47.177846   290 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task3eSafg/build/asan/bin/kudu with pid 1790
I20261118 08:54:48.164377   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--webserver_interface=127.0.72.189
--webserver_port=35929
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 08:54:48.501622  2458 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:48.501938  2458 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:48.502033  2458 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:48.510529  2458 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:48.510651  2458 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:48.510743  2458 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:48.510811  2458 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:48.520977  2458 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=35929
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:48.522761  2458 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:48.524560  2458 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:48.533233  2469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:48.533429  2470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:48.533483  2472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:48.534878  2458 server_base.cc:1061] running on GCE node
I20261118 08:54:48.535692  2458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:48.537107  2458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:48.538306  2458 hybrid_clock.cc:648] HybridClock initialized: now 1794992088538248 us; error 50 us; skew 500 ppm
I20261118 08:54:48.538630  2458 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:48.540664  2458 webserver.cc:577] Webserver started at http://127.0.72.189:35929/ using document root <none> and password file <none>
I20261118 08:54:48.541405  2458 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:48.541508  2458 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:48.546828  2458 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.005s	sys 0.000s
I20261118 08:54:48.549556  2478 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:48.550855  2458 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20261118 08:54:48.551024  2458 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
uuid: "857a577b88ba46cc846ecd1a25a722ca"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:48.551762  2458 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:48.598577  2458 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:48.599639  2458 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:48.599933  2458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:48.620504  2458 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:33939
I20261118 08:54:48.620548  2529 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:33939 every 8 connection(s)
I20261118 08:54:48.622169  2458 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
I20261118 08:54:48.622855   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 2458
I20261118 08:54:48.630662  2530 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:48.635027  2530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap starting.
W20261118 08:54:48.641742  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 6: this message will repeat every 5th retry.
I20261118 08:54:48.647533  1258 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:48.652526  2530 log.cc:826] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:48.700654  2530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=54 overwritten=0 applied=54 ignored=0} inserts{seen=36 ignored=0} mutations{seen=45 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 08:54:48.701287  2530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap complete.
I20261118 08:54:48.706863  2530 raft_consensus.cc:360] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:48.707620  2530 raft_consensus.cc:748] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857a577b88ba46cc846ecd1a25a722ca, State: Initialized, Role: FOLLOWER
I20261118 08:54:48.708285  2530 consensus_queue.cc:261] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 54, Last appended: 6.54, Last appended by leader: 54, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:48.709052  2534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:48.709292  2534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:48.710232  2530 sys_catalog.cc:565] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:48.722970  2548 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:48.723174  2548 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: loading cluster ID for follower catalog manager: success
I20261118 08:54:48.728024  2548 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: acquiring CA information for follower catalog manager: success
I20261118 08:54:48.731308  2548 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:48.750785  2495 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:41845; Asking this server to re-register.
I20261118 08:54:48.751885  1258 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:48.736522  2458 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:54:48.752414  2458 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:48.752772   985 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:48.754333  2495 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:48.755697  2494 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:48487; Asking this server to re-register.
I20261118 08:54:48.756332   985 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:48.757711  2494 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:48.806824  1123 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:48.808385  2494 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:48955; Asking this server to re-register.
I20261118 08:54:48.809010  1123 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:48.810353  2494 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:54:49.143632  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 6: this message will repeat every 5th retry.
W20261118 08:54:49.320322  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 6: this message will repeat every 5th retry.
I20261118 08:54:49.321818  2561 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:49.322079  2561 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:49.323889  2561 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:49.324710  2480 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:49.325904  2479 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:49.326093  2480 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:49.326259  2480 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:49.326573  2561 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20261118 08:54:49.810585  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 6: this message will repeat every 5th retry.
I20261118 08:54:49.933621  2567 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:49.933812  2567 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:49.934547  2567 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:49.938050  2480 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:49.938326  2479 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:49.938469  2479 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:49.938804  2567 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 08:54:50.522322  2581 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:50.522505  2581 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:50.523126  2581 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:50.526448  2480 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:50.526791  2479 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:50.526935  2479 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:50.527194  2581 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 08:54:50.824267  2242 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:50.824481  2242 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:50.825088  2242 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:54:50.826112  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:54:50.826280  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:54:50.826400  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:50.826656  2242 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:54:51.078720   290 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task3eSafg/build/asan/bin/kudu pid 2458
I20261118 08:54:51.086760  2525 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 08:54:51.192257  2598 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:51.192257  2593 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:51.192411   290 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task3eSafg/build/asan/bin/kudu with pid 2458
I20261118 08:54:51.192440  2593 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:54:51.193184  2593 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:54:51.194191  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:54:51.194352  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:54:51.194466  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:51.194710  2593 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:54:51.210144  1010 connection.cc:573] client connection to 127.0.72.189:33939 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:51.210884  1264 connection.cc:573] client connection to 127.0.72.189:33939 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
I20261118 08:54:52.007283   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--webserver_interface=127.0.72.189
--webserver_port=35929
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 08:54:52.356608  2617 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:52.356947  2617 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:52.357081  2617 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:52.365967  2617 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:52.366067  2617 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:52.366137  2617 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:52.366214  2617 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:52.377364  2617 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.189:33939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.72.189
--webserver_port=35929
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:52.379067  2617 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:52.380987  2617 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:52.389887  2629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:52.389856  2632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:52.389817  2630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:52.392782  2617 server_base.cc:1061] running on GCE node
I20261118 08:54:52.393589  2617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:52.395141  2617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:52.396345  2617 hybrid_clock.cc:648] HybridClock initialized: now 1794992092396276 us; error 69 us; skew 500 ppm
I20261118 08:54:52.396692  2617 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:52.398834  2617 webserver.cc:577] Webserver started at http://127.0.72.189:35929/ using document root <none> and password file <none>
I20261118 08:54:52.399610  2617 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:52.399724  2617 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:52.405570  2617 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20261118 08:54:52.408568  2638 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:52.409999  2617 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20261118 08:54:52.410172  2617 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
uuid: "857a577b88ba46cc846ecd1a25a722ca"
format_stamp: "Formatted at 2026-11-18 08:54:22 on dist-test-slave-m8c8"
I20261118 08:54:52.410979  2617 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:52.423744  2617 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:52.424645  2617 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:52.424940  2617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:52.445588  2617 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.189:33939
I20261118 08:54:52.445605  2690 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.189:33939 every 8 connection(s)
I20261118 08:54:52.447413  2617 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-1/data/info.pb
I20261118 08:54:52.454308  2691 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:52.456516   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 2617
I20261118 08:54:52.458003  2691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap starting.
I20261118 08:54:52.472829  2691 log.cc:826] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:52.515352  2691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=54 overwritten=0 applied=54 ignored=0} inserts{seen=36 ignored=0} mutations{seen=45 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 08:54:52.515925  2691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: Bootstrap complete.
I20261118 08:54:52.521214  2691 raft_consensus.cc:360] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:52.522043  2691 raft_consensus.cc:748] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857a577b88ba46cc846ecd1a25a722ca, State: Initialized, Role: FOLLOWER
I20261118 08:54:52.522727  2691 consensus_queue.cc:261] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 54, Last appended: 6.54, Last appended by leader: 54, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:52.523522  2695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:52.523782  2695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:52.524540  2691 sys_catalog.cc:565] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:52.537729  2709 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:52.537936  2709 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: loading cluster ID for follower catalog manager: success
I20261118 08:54:52.542081  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:54:52.547641  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:52.547996  2709 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: acquiring CA information for follower catalog manager: success
I20261118 08:54:52.550210  2709 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:52.681993  1258 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:52.685280  2656 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:55959; Asking this server to re-register.
I20261118 08:54:52.686005  1258 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:52.687801  2656 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:52.763193   985 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:52.764703  2656 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:42005; Asking this server to re-register.
I20261118 08:54:52.765357   985 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:52.766754  2656 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:52.862286  1123 heartbeater.cc:345] Connected to a master server at 127.0.72.189:33939
I20261118 08:54:52.863750  2656 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:43047; Asking this server to re-register.
I20261118 08:54:52.864310  1123 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:52.865576  2656 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:53.106848  2718 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:53.107143  2718 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:53.109498  2718 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:53.110328  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:53.111881  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:53.112092  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:53.112221  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:53.112576  2718 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 08:54:53.803686  2732 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:53.803856  2732 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:53.804476  2732 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:53.806480  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:53.806631  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:53.806768  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:53.807036  2732 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20261118 08:54:54.210915  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 6: this message will repeat every 5th retry.
I20261118 08:54:54.676335  2744 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:54.676558  2744 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:54.677285  2744 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:54.679185  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:54.679455  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:54.679593  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:54.679852  2744 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20261118 08:54:55.052481  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 6: this message will repeat every 5th retry.
I20261118 08:54:55.641901  2764 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:55.642112  2764 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:55.642870  2764 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:55.643115   290 external_mini_cluster.cc:1372] Running /tmp/dist-test-task3eSafg/build/asan/bin/kudu
/tmp/dist-test-task3eSafg/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--webserver_interface=127.0.72.190
--webserver_port=41231
--builtin_ntp_servers=127.0.72.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 08:54:55.645220  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:55.645453  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:55.645566  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:55.645820  2764 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20261118 08:54:55.983181  2768 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 08:54:55.983562  2768 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 08:54:55.983675  2768 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 08:54:55.993059  2768 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 08:54:55.993150  2768 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 08:54:55.993213  2768 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 08:54:55.993279  2768 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 08:54:56.003607  2768 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.72.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:45085
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.72.190:42341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.72.190
--webserver_port=41231
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 08:43:23 UTC on bdcb31816ec0
build id 14841
ASAN enabled
I20261118 08:54:56.005210  2768 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 08:54:56.007289  2768 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 08:54:56.015755  2779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:56.015780  2780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 08:54:56.016764  2782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 08:54:56.017274  2768 server_base.cc:1061] running on GCE node
I20261118 08:54:56.017982  2768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 08:54:56.019335  2768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 08:54:56.020569  2768 hybrid_clock.cc:648] HybridClock initialized: now 1794992096020505 us; error 52 us; skew 500 ppm
I20261118 08:54:56.020874  2768 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 08:54:56.022970  2768 webserver.cc:577] Webserver started at http://127.0.72.190:41231/ using document root <none> and password file <none>
I20261118 08:54:56.023710  2768 fs_manager.cc:362] Metadata directory not provided
I20261118 08:54:56.023805  2768 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 08:54:56.029352  2768 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20261118 08:54:56.032338  2788 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 08:54:56.033831  2768 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20261118 08:54:56.034003  2768 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data,/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
uuid: "78d4026bc230421ea10b8c68a9ef7871"
format_stamp: "Formatted at 2026-11-18 08:54:21 on dist-test-slave-m8c8"
I20261118 08:54:56.034749  2768 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 08:54:56.049816  2768 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 08:54:56.050731  2768 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 08:54:56.050995  2768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 08:54:56.069643  2768 rpc_server.cc:307] RPC server started. Bound to: 127.0.72.190:42341
I20261118 08:54:56.069659  2839 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.72.190:42341 every 8 connection(s)
I20261118 08:54:56.071275  2768 server_base.cc:1193] Dumped server information to /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/master-0/data/info.pb
I20261118 08:54:56.071602   290 external_mini_cluster.cc:1434] Started /tmp/dist-test-task3eSafg/build/asan/bin/kudu as pid 2768
I20261118 08:54:56.079089  2840 sys_catalog.cc:263] Verifying existing consensus state
I20261118 08:54:56.083700  2840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap starting.
I20261118 08:54:56.101110  2840 log.cc:826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Log is configured to *not* fsync() on all Append() calls
I20261118 08:54:56.141664  2840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap replayed 1/1 log segments. Stats: ops{read=54 overwritten=0 applied=54 ignored=0} inserts{seen=36 ignored=0} mutations{seen=45 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 08:54:56.142185  2840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Bootstrap complete.
I20261118 08:54:56.149351  2840 raft_consensus.cc:360] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:56.150352  2840 raft_consensus.cc:748] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Initialized, Role: FOLLOWER
I20261118 08:54:56.150998  2840 consensus_queue.cc:261] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 54, Last appended: 6.54, Last appended by leader: 54, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:56.151835  2847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:56.152213  2847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:56.153226  2840 sys_catalog.cc:565] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 08:54:56.153894  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:56.165248  2861 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:56.165446  2861 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: loading cluster ID for follower catalog manager: success
I20261118 08:54:56.170416  2861 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: acquiring CA information for follower catalog manager: success
I20261118 08:54:56.173664  2861 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 08:54:56.201002  2768 master_runner.cc:187] Error getting master registration for 127.0.72.188:41259: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111), 
I20261118 08:54:56.204668  2768 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:56.273921   984 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:56.274189   984 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
I20261118 08:54:56.276432  2805 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "ad2789ff5b54489c84966cff78be4721" instance_seqno: 1794992064082361) as {username='slave'} at 127.0.72.129:54409; Asking this server to re-register.
I20261118 08:54:56.277091   984 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:56.278643  2805 ts_manager.cc:210] Registered new tserver with Master: ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
I20261118 08:54:56.515724  1257 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:56.517786  2805 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" instance_seqno: 1794992065071421) as {username='slave'} at 127.0.72.131:43045; Asking this server to re-register.
I20261118 08:54:56.518414  1257 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:56.519778  2805 ts_manager.cc:210] Registered new tserver with Master: 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
I20261118 08:54:56.657977  1122 heartbeater.cc:345] Connected to a master server at 127.0.72.190:42341
I20261118 08:54:56.659647  2805 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" instance_seqno: 1794992064582482) as {username='slave'} at 127.0.72.130:36031; Asking this server to re-register.
I20261118 08:54:56.660270  1122 heartbeater.cc:462] Registering TS with master...
I20261118 08:54:56.661495  2805 ts_manager.cc:210] Registered new tserver with Master: 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
I20261118 08:54:56.696574  2874 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:56.696913  2874 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:56.698781  2874 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:56.699431  2789 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:56.699493  2666 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 7 candidate_status { last_received { term: 6 index: 54 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca" is_pre_election: true
I20261118 08:54:56.699885  2666 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 6.
I20261118 08:54:56.700531  2790 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 
I20261118 08:54:56.700898  2874 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20261118 08:54:56.700930  2789 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:56.701074  2874 raft_consensus.cc:499] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 08:54:56.701180  2874 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 6 FOLLOWER]: Advancing to term 7
I20261118 08:54:56.704665  2874 raft_consensus.cc:521] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:56.705256  2874 leader_election.cc:290] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 7 election: Requested vote from peers 857a577b88ba46cc846ecd1a25a722ca (127.0.72.189:33939), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
I20261118 08:54:56.705741  2666 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78d4026bc230421ea10b8c68a9ef7871" candidate_term: 7 candidate_status { last_received { term: 6 index: 54 } } ignore_live_leader: false dest_uuid: "857a577b88ba46cc846ecd1a25a722ca"
I20261118 08:54:56.705956  2666 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 6 FOLLOWER]: Advancing to term 7
W20261118 08:54:56.706575  2789 leader_election.cc:336] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:56.709594  2666 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78d4026bc230421ea10b8c68a9ef7871 in term 7.
I20261118 08:54:56.710006  2790 leader_election.cc:304] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78d4026bc230421ea10b8c68a9ef7871, 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1
I20261118 08:54:56.710306  2874 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 7 FOLLOWER]: Leader election won for term 7
I20261118 08:54:56.710738  2874 raft_consensus.cc:705] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [term 7 LEADER]: Becoming Leader. State: Replica: 78d4026bc230421ea10b8c68a9ef7871, State: Running, Role: LEADER
I20261118 08:54:56.711311  2874 consensus_queue.cc:238] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 54, Committed index: 54, Last appended: 6.54, Last appended by leader: 54, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:56.714267  2876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 7 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:56.714550  2876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:56.716113  2666 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Refusing update from remote peer 78d4026bc230421ea10b8c68a9ef7871: Log matching property violated. Preceding OpId in replica: term: 6 index: 54. Preceding OpId from leader: term: 7 index: 55. (index mismatch)
W20261118 08:54:56.716861  2789 consensus_peers.cc:597] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 -> Peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Couldn't send request to peer 6c7d9f52503f41e4bafe2821e6fd6af1. Status: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:56.717177  2878 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 55, Last known committed idx: 54, Time since last communication: 0.000s
I20261118 08:54:56.721215  2881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78d4026bc230421ea10b8c68a9ef7871. Latest consensus state: current_term: 7 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:56.721450  2881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:56.721443  2880 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 08:54:56.722764  2874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:56.723037  2874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871 [sys.catalog]: This master's current role is: LEADER
I20261118 08:54:56.724663  2880 catalog_manager.cc:694] Loaded metadata for table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30]
I20261118 08:54:56.725175  2880 catalog_manager.cc:694] Loaded metadata for table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a]
I20261118 08:54:56.725682  2880 catalog_manager.cc:694] Loaded metadata for table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70]
I20261118 08:54:56.726058  2880 catalog_manager.cc:694] Loaded metadata for table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a]
I20261118 08:54:56.726359  2880 catalog_manager.cc:694] Loaded metadata for table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2]
I20261118 08:54:56.726704  2880 catalog_manager.cc:694] Loaded metadata for table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b]
I20261118 08:54:56.726996  2880 catalog_manager.cc:694] Loaded metadata for table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7]
I20261118 08:54:56.727305  2880 catalog_manager.cc:694] Loaded metadata for table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957]
I20261118 08:54:56.727635  2880 catalog_manager.cc:694] Loaded metadata for table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]
I20261118 08:54:56.729702  2880 tablet_loader.cc:96] loaded metadata for tablet 368ffd5a7d724f47abeea8f223415eac (table default.table_1ee6cacf31104df284b253928039759b [id=1950821cc4b84c97939b8da6baaab97a])
I20261118 08:54:56.730134  2880 tablet_loader.cc:96] loaded metadata for tablet 3c8c4245c7ac4d8ca2e7fa521b6d49d3 (table default.table_bc1f66261ac64a87912e19a32ce0ca44 [id=4fcf984156584e30aac26283f71adda2])
I20261118 08:54:56.730571  2880 tablet_loader.cc:96] loaded metadata for tablet 48a6e2d96fe442b6a84b4773993b4f54 (table default.table_567d34233a4b440a952301f19a6bcb10 [id=39ea4f28a4744d1290fe1e9246789b0a])
I20261118 08:54:56.730955  2880 tablet_loader.cc:96] loaded metadata for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf (table default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8])
I20261118 08:54:56.731254  2880 tablet_loader.cc:96] loaded metadata for tablet 50744c2d59124a22ac0a76f2f8e60974 (table default.table_294f03c6875c4da0845888273ea80a2a [id=0146c51b5d004bd6b63f5e1c9e1dcf30])
I20261118 08:54:56.731870  2880 tablet_loader.cc:96] loaded metadata for tablet 748947f292334e7a97abfa599634006d (table default.table_c3f2df1b938946c19c1ca00d8fcbd640 [id=812c3949fc5544c09b17567b3e81977b])
I20261118 08:54:56.731993  2881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "78d4026bc230421ea10b8c68a9ef7871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } } }
I20261118 08:54:56.732201  2881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [sys.catalog]: This master's current role is: FOLLOWER
I20261118 08:54:56.732506  2880 tablet_loader.cc:96] loaded metadata for tablet 86bf01af75674f5491b024ac43a063b0 (table default.table_5f1fb0758cf1413eaed1d8b4717aa902 [id=c2ea63be7ece4af4bcfb932bd174e957])
I20261118 08:54:56.733342  2880 tablet_loader.cc:96] loaded metadata for tablet d2ce64bc39634b5882b1beaa68e66cbe (table default.table_41d62c0cf6b54938b8e1770b859e56e8 [id=a57cd9d759e94822ae1e9ecb7faad9f7])
I20261118 08:54:56.733637  2880 tablet_loader.cc:96] loaded metadata for tablet d546b7f20a7f4d3b9e5612080df04be6 (table default.table_ef26d2c593994e118ff3d01a0f5a9a1d [id=19b3ec381a984304badca62634eadd70])
I20261118 08:54:56.734351  2880 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 08:54:56.735373  2880 catalog_manager.cc:1310] Loaded cluster ID: 0033b87516ad4c4a82378eedc366b8d8
I20261118 08:54:56.735484  2880 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 08:54:56.736742  2880 catalog_manager.cc:1555] Loading token signing keys...
I20261118 08:54:56.737569  2880 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Loaded TSK: 0
I20261118 08:54:56.738339  2880 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 08:54:56.739204  2880 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 08:54:56.740087  2880 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 08:54:56.762379  2804 master_service.cc:965] ReplaceTablet: received request to replace tablet 3e292319f0c44e34b49ae6d579df2ecb from {username='slave'} at 127.0.0.1:38312
I20261118 08:54:56.762530  2803 master_service.cc:965] ReplaceTablet: received request to replace tablet 3e292319f0c44e34b49ae6d579df2ecb from {username='slave'} at 127.0.0.1:38312
I20261118 08:54:56.762781  2801 master_service.cc:965] ReplaceTablet: received request to replace tablet 48a6e2d96fe442b6a84b4773993b4f54 from {username='slave'} at 127.0.0.1:38312
I20261118 08:54:56.762859  2805 master_service.cc:965] ReplaceTablet: received request to replace tablet 48a6e2d96fe442b6a84b4773993b4f54 from {username='slave'} at 127.0.0.1:38312
I20261118 08:54:56.763944  2803 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38312:
table { table_name: "default.Table_d2d73a819dc64fe0badbe6d84293bfff" } modify_external_catalogs: true
I20261118 08:54:56.764132  2802 master_service.cc:965] ReplaceTablet: received request to replace tablet 4eecc49bfe014d1d86c84e8a4f16a5bf from {username='slave'} at 127.0.0.1:38312
I20261118 08:54:56.764181  2797 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38312:
table { table_name: "default.Table_41d62c0cf6b54938b8e1770b859e56e8" } new_table_name: "default.Table_eefa15fa899c4b1e9162efce0db7e4af" modify_external_catalogs: true
I20261118 08:54:56.764325  2803 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38312:
table { table_name: "default.Table_d2d73a819dc64fe0badbe6d84293bfff" } modify_external_catalogs: true
I20261118 08:54:56.764467  2796 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38312:
table { table_name: "default.Table_5f1fb0758cf1413eaed1d8b4717aa902" } new_table_name: "default.Table_d9954c97f0054586ab7cf9e5e98b3fa4" modify_external_catalogs: true
I20261118 08:54:56.763722  2804 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38312:
name: "default.Table_8f6feb81c1374d42bc1e0acb856b75cd"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:56.765012  2798 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:38312:
table { table_name: "default.Table_bc1f66261ac64a87912e19a32ce0ca44" } new_table_name: "default.Table_7fe6a97ebd294ce68601eaf9f252d0ee" modify_external_catalogs: true
I20261118 08:54:56.765517  2800 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38312:
name: "default.Table_82d1541f550a4d5482cff56810c97a22"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:56.763718  2799 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38312:
name: "default.Table_0327d29a2f19450b872c54c554d7654e"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
F20261118 08:54:56.772107  2860 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 1, previous 1; dumping first 2 out of 10 received events: NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_bc1f66261ac64a87912e19a32ce0ca44, message={"server":"","servicePrincipal":"","db":"default","table":"table_bc1f66261ac64a87912e19a32ce0ca44","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_bc1f66261ac64a87912e19a32ce0ca44\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_bc1f66261ac64a87912e19a32ce0ca44\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_bc1f66261ac64a87912e19a32ce0ca44\",\"kudu.table_id\":\"4fcf984156584e30aac26283f71adda2\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=1, eventTime=1794992065, eventType=CREATE_TABLE, dbName=default, tableName=table_8b08d3e4ed824f1680377a6d5e7518a8, message={"server":"","servicePrincipal":"","db":"default","table":"table_8b08d3e4ed824f1680377a6d5e7518a8","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794992065},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/warehouse/table_8b08d3e4ed824f1680377a6d5e7518a8\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_8b08d3e4ed824f1680377a6d5e7518a8\",\"kudu.table_id\":\"b42eb739941947f8b7d79fddd923eb52\",\"transient_lastDdlTime\":\"1794992065\",\"comment\":\"\",\"kudu.master_addresses\":\"127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"0033b87516ad4c4a82378eedc366b8d8\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794992065,"files":[]}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
    @     0x7fc67a688d5d  google::LogMessage::Fail() at ??:0
    @     0x7fc67a68cb23  google::LogMessage::SendToLog() at ??:0
    @     0x7fc67a68875c  google::LogMessage::Flush() at ??:0
I20261118 08:54:56.779068  2802 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 4eecc49bfe014d1d86c84e8a4f16a5bf
    @     0x7fc67a6889a1  google::LogMessage::~LogMessage() at ??:0
I20261118 08:54:56.781347  2802 catalog_manager.cc:6936] ReplaceTablet: tablet 4eecc49bfe014d1d86c84e8a4f16a5bf deleted and replaced by tablet 380fa8ff3e2f428b81b7567bc7bcefee
    @     0x7fc68fb089a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 08:54:56.801615  2801 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 78d4026bc230421ea10b8c68a9ef7871: Sending DeleteTablet for 3 replicas of tablet 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:54:56.802258  2801 catalog_manager.cc:6936] ReplaceTablet: tablet 48a6e2d96fe442b6a84b4773993b4f54 deleted and replaced by tablet 625ed666c1f748a2a6fd4fbb44746496
    @     0x7fc68fb07251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
    @     0x7fc68fb0eecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 08:54:56.813431   915 tablet_service.cc:1547] Processing DeleteTablet for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf with delete_type TABLET_DATA_DELETED (replaced by tablet 380fa8ff3e2f428b81b7567bc7bcefee) from {username='slave'} at 127.0.0.1:55682
I20261118 08:54:56.813871   916 tablet_service.cc:1547] Processing DeleteTablet for tablet 48a6e2d96fe442b6a84b4773993b4f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 625ed666c1f748a2a6fd4fbb44746496) from {username='slave'} at 127.0.0.1:55682
I20261118 08:54:56.815179  1053 tablet_service.cc:1547] Processing DeleteTablet for tablet 4eecc49bfe014d1d86c84e8a4f16a5bf with delete_type TABLET_DATA_DELETED (replaced by tablet 380fa8ff3e2f428b81b7567bc7bcefee) from {username='slave'} at 127.0.0.1:45156
I20261118 08:54:56.815277  1054 tablet_service.cc:1547] Processing DeleteTablet for tablet 48a6e2d96fe442b6a84b4773993b4f54 with delete_type TABLET_DATA_DELETED (replaced by tablet 625ed666c1f748a2a6fd4fbb44746496) from {username='slave'} at 127.0.0.1:45156
I20261118 08:54:56.819032  2802 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38312:
name: "default.Table_bfd124a61a904e60bc99bee8716f64cc"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
    @     0x7fc68fb0ed49  std::_Function_handler<>::_M_invoke() at ??:0
I20261118 08:54:56.819875  2805 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38312:
name: "default.Table_52bba9536ae942a59235d8087d61576b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 08:54:56.823089  2801 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38312:
name: "default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 08:54:56.824566  2898 tablet_replica.cc:333] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:56.824776  2898 raft_consensus.cc:2261] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:56.825012  2898 raft_consensus.cc:2290] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
    @     0x7fc67bad1dc9  kudu::Thread::SuperviseThread() at ??:0
I20261118 08:54:56.826103  2896 tablet_replica.cc:333] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:56.826390  2896 raft_consensus.cc:2261] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:56.826634  2896 raft_consensus.cc:2290] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:56.827246  2896 ts_tablet_manager.cc:1962] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
    @     0x7fc67f1ac6db  start_thread at ??:0
I20261118 08:54:56.829713  2898 ts_tablet_manager.cc:1962] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
    @     0x7fc677c1c61f  clone at ??:0
I20261118 08:54:56.829880  1053 tablet_service.cc:1500] Processing CreateTablet for tablet 380fa8ff3e2f428b81b7567bc7bcefee (DEFAULT_TABLE table=default.table_d2d73a819dc64fe0badbe6d84293bfff [id=fa8c2c5f6be14649a1bd9abaca0167c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 08:54:56.830518  1053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380fa8ff3e2f428b81b7567bc7bcefee. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 08:54:56.835709  2896 ts_tablet_manager.cc:1975] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:56.835860  2896 log.cc:1199] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/4eecc49bfe014d1d86c84e8a4f16a5bf
I20261118 08:54:56.836304  2896 ts_tablet_manager.cc:1996] T 4eecc49bfe014d1d86c84e8a4f16a5bf P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:56.837409  2898 ts_tablet_manager.cc:1975] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:56.837527  2898 log.cc:1199] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/4eecc49bfe014d1d86c84e8a4f16a5bf
I20261118 08:54:56.837800  2896 tablet_replica.cc:333] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: stopping tablet replica
I20261118 08:54:56.837947  2898 ts_tablet_manager.cc:1996] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:56.838035  2896 raft_consensus.cc:2261] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 1 LEADER]: Raft consensus shutting down.
I20261118 08:54:56.838387  2896 raft_consensus.cc:2290] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:56.839061  2896 ts_tablet_manager.cc:1962] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:56.839090  2898 tablet_replica.cc:333] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: stopping tablet replica
I20261118 08:54:56.839286  2898 raft_consensus.cc:2261] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 08:54:56.839473  2898 raft_consensus.cc:2290] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 08:54:56.840044  2898 ts_tablet_manager.cc:1962] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 08:54:56.845880  2896 ts_tablet_manager.cc:1975] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:56.846045  2896 log.cc:1199] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-0/wal/wals/48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:54:56.846503  2896 ts_tablet_manager.cc:1996] T 48a6e2d96fe442b6a84b4773993b4f54 P ad2789ff5b54489c84966cff78be4721: Deleting consensus metadata
I20261118 08:54:56.847316  2900 tablet_bootstrap.cc:492] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042: Bootstrap starting.
I20261118 08:54:56.848860  2898 ts_tablet_manager.cc:1975] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 08:54:56.848987  2898 log.cc:1199] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: Deleting WAL directory at /tmp/dist-test-task3eSafg/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794992047095830-290-0/minicluster-data/ts-1/wal/wals/48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:54:56.849325  2900 tablet_bootstrap.cc:654] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042: Neither blocks nor log segments found. Creating new log.
I20261118 08:54:56.849431  2898 ts_tablet_manager.cc:1996] T 48a6e2d96fe442b6a84b4773993b4f54 P 213b5f7a64e74b109a3a9387e5975042: Deleting consensus metadata
I20261118 08:54:56.851269  2900 tablet_bootstrap.cc:492] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042: No bootstrap required, opened a new log
I20261118 08:54:56.851366  2900 ts_tablet_manager.cc:1449] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042: Time spent bootstrapping tablet: real 0.004s	user 0.003s	sys 0.000s
I20261118 08:54:56.852097  2900 raft_consensus.cc:360] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:56.852237  2900 raft_consensus.cc:386] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 08:54:56.852293  2900 raft_consensus.cc:748] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b5f7a64e74b109a3a9387e5975042, State: Initialized, Role: FOLLOWER
I20261118 08:54:56.852483  2900 consensus_queue.cc:261] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:56.854107  2900 ts_tablet_manager.cc:1480] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
W20261118 08:54:56.856462  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 1: this message will repeat every 5th retry.
W20261118 08:54:56.913125  1124 tablet.cc:2441] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 08:54:56.952610  1257 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
W20261118 08:54:56.999491  1009 connection.cc:573] server connection from 127.0.0.1:45156 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:56.999507   873 connection.cc:573] server connection from 127.0.0.1:55682 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:56.999794  1145 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:56.999658  2895 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 08:54:56.810971 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:44234 (local address 127.0.72.131:46447)
1118 08:54:56.812065 (+  1094us) server_negotiation.cc:207] Beginning negotiation
1118 08:54:56.812072 (+     7us) server_negotiation.cc:400] Waiting for connection header
1118 08:54:56.812102 (+    30us) server_negotiation.cc:408] Connection header received
1118 08:54:56.812200 (+    98us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
1118 08:54:56.812207 (+     7us) server_negotiation.cc:462] Received NEGOTIATE request from client
1118 08:54:56.812292 (+    85us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
1118 08:54:56.812571 (+   279us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
1118 08:54:56.816151 (+  3580us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
1118 08:54:56.820721 (+  4570us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
1118 08:54:56.999599 (+178878us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44234: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":283,"threads_started":1}
W20261118 08:54:57.000103  1010 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:57.000169  1264 connection.cc:573] client connection to 127.0.72.190:42341 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 08:54:57.000890  1264 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.72.190:42341)
W20261118 08:54:57.001286  1264 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:57.001581  1264 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:57.001785  1264 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.72.190:42341)
W20261118 08:54:57.130183  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 1: this message will repeat every 5th retry.
I20261118 08:54:57.294582   984 heartbeater.cc:500] Master 127.0.72.190:42341 was elected leader, sending a full tablet report...
I20261118 08:54:57.324908  2914 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:57.325142  2914 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:57.325933  2914 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:57.328076  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:57.541000  2911 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:54:57.541224  2911 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:57.541880  2911 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:54:57.543037  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:54:57.543243  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:54:57.543370  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:57.543655  2911 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:54:58.164122  2922 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:58.164320  2922 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:58.164953  2922 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:54:58.165958  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:54:58.166116  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:54:58.166230  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:58.166474  2922 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:54:58.339927  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:58.340075  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:58.340684  2930 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:54:58.340719  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:54:58.340936  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:54:58.579646   290 master-stress-test.cc:579] Killing test thread 1/25
W20261118 08:54:58.765995  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 11: this message will repeat every 5th retry.
W20261118 08:54:58.925338  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 11: this message will repeat every 5th retry.
I20261118 08:54:58.970541  2935 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:58.970737  2935 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:58.971525  2935 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:58.972391  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20261118 08:54:58.973538  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:54:58.973683  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:54:58.973835  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:58.974081  2935 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:54:59.513338  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 11: this message will repeat every 5th retry.
I20261118 08:54:59.533136  2922 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:54:59.533301  2922 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:54:59.534049  2922 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:54:59.534900  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:54:59.535069  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:54:59.535203  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:54:59.535462  2922 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:54:59.625959  2943 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:54:59.626154  2943 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:54:59.626849  2943 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:54:59.628777  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:54:59.628944  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:54:59.629087  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:54:59.629323  2943 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:54:59.872606  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 11: this message will repeat every 5th retry.
I20261118 08:55:00.480723  2924 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:00.480903  2924 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:55:00.481573  2924 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:55:00.482386  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:55:00.482545  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:55:00.482679  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:00.482923  2924 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:55:00.958114  2956 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:00.958333  2956 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:00.959080  2956 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:00.961369  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:55:00.961509  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:55:00.961660  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:00.961908  2956 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:55:01.603644  2967 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:55:01.603856  2967 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:01.604725  2967 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:55:01.605660  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:55:01.605860  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:55:01.606006  1012 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:01.606235  2967 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:55:02.074290  2970 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:02.074482  2970 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:02.075208  2970 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:02.077728  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:55:02.077939  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:55:02.078099  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:02.078395  2970 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:55:03.289548  2923 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:03.289760  2923 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:03.290491  2923 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:03.291337  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:55:03.291550  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:55:03.291647  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:03.291880  2923 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:55:03.342417  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:03.342625  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:55:04.198098  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 11: this message will repeat every 5th retry.
W20261118 08:55:05.403669  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 11: this message will repeat every 5th retry.
I20261118 08:55:05.482529  3014 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:05.482851  3014 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:05.483740  3014 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:05.484767  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20261118 08:55:05.485849  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:55:05.486025  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:55:05.486156  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:05.486399  3014 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:55:05.703622  3013 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:55:05.703832  3013 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:05.704566  3013 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:55:05.705437  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:55:05.705590  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:55:05.705739  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:05.705971  3013 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20261118 08:55:07.225171  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 6: this message will repeat every 5th retry.
W20261118 08:55:07.998588  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 16: this message will repeat every 5th retry.
W20261118 08:55:08.118414  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 6: this message will repeat every 5th retry.
I20261118 08:55:08.344391  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:08.344623  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:55:09.087571  3009 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:09.087778  3009 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:09.088557  3009 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:09.089561  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:55:09.089715  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:55:09.089874  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:09.090145  3009 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:09.165706  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 16: this message will repeat every 5th retry.
W20261118 08:55:09.360040  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 16: this message will repeat every 5th retry.
W20261118 08:55:09.896785  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 16: this message will repeat every 5th retry.
I20261118 08:55:09.960031  3051 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:09.960263  3051 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:09.961004  3051 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:09.963049  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:55:09.963208  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:55:09.963357  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:09.963650  3051 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:55:10.621586  3009 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:10.621821  3009 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:10.622414  3009 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:10.623314  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:55:10.623510  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:55:10.623628  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:10.623854  3009 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:55:12.581561  3076 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:55:12.581813  3076 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:12.582634  3076 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:55:12.583623  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:55:12.583786  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:55:12.583930  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:12.584182  3076 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:55:12.617372  3077 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:12.617573  3077 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:12.618295  3077 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:12.619261  2639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 08:55:12.620399  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:55:12.620559  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:55:12.620682  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:12.620905  3077 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:55:12.704136  3009 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:12.704350  3009 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:55:12.705029  3009 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:55:12.706007  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:55:12.706167  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:55:12.706311  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:12.706550  3009 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:55:13.345884  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:13.346053  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:55:13.436771  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 16: this message will repeat every 5th retry.
I20261118 08:55:14.636415  3009 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:14.636583  3009 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:14.637305  3009 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:14.638192  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:55:14.638337  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:55:14.638448  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:14.638674  3009 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:14.873409  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 16: this message will repeat every 5th retry.
I20261118 08:55:16.674403  3110 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:16.674600  3110 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:55:16.675334  3110 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:55:16.676064  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:55:16.676261  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:55:16.676383  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:16.676620  3110 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:16.852854  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 11: this message will repeat every 5th retry.
W20261118 08:55:17.424782  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 11: this message will repeat every 5th retry.
W20261118 08:55:18.037348  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 21: this message will repeat every 5th retry.
I20261118 08:55:18.347512  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:18.347718  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:55:19.310201  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 21: this message will repeat every 5th retry.
W20261118 08:55:19.495365  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 21: this message will repeat every 5th retry.
W20261118 08:55:20.152040  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 21: this message will repeat every 5th retry.
I20261118 08:55:21.214655  3147 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:55:21.214861  3147 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:21.215817  3147 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:55:21.216738  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:55:21.216929  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:55:21.217072  1012 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:21.217336  3147 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:55:22.421103  3158 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:22.421324  3158 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:22.422132  3158 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:22.423287  2639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:55:22.424746  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:55:22.424921  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:55:22.425053  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:22.425354  3158 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:55:23.290721  2998 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:23.290930  2998 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:23.291654  2998 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:23.292619  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:55:23.292770  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:55:23.292913  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:23.293151  2998 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:55:23.349212  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:23.349423  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:55:23.535778  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 21: this message will repeat every 5th retry.
W20261118 08:55:25.075994  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 21: this message will repeat every 5th retry.
W20261118 08:55:25.620873  1256 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (343 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:55:25.695674  1122 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42341 (72 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42341: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:55:25.846400   983 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (146 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:55:26.376350  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 16: this message will repeat every 5th retry.
I20261118 08:55:26.647089  3110 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:26.647313  3110 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:26.648067  3110 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:26.648962  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:55:26.649140  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:55:26.649291  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:26.649569  3110 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:26.769910  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 16: this message will repeat every 5th retry.
W20261118 08:55:27.285463  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 26: this message will repeat every 5th retry.
I20261118 08:55:28.350706  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:28.350883  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:55:28.431332  3110 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:28.431565  3110 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:28.432268  3110 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:28.433226  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:55:28.433385  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:55:28.433503  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:28.433722  3110 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:55:29.222297  3220 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:29.222513  3220 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:29.223501  3220 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:29.224879  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:55:29.226145  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:55:29.226372  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:55:29.226468  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:29.226708  3220 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:55:29.710383  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 26: this message will repeat every 5th retry.
W20261118 08:55:30.112123  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 26: this message will repeat every 5th retry.
W20261118 08:55:30.612264  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 26: this message will repeat every 5th retry.
I20261118 08:55:31.589365  3182 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:31.589563  3182 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:31.590410  3182 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:31.591483  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:55:31.591630  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:55:31.591786  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:31.592043  3182 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:55:33.352298  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:33.352494  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:55:33.658567  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 26: this message will repeat every 5th retry.
I20261118 08:55:34.941478  3242 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:34.941672  3242 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:55:34.942396  3242 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:55:34.943277  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:55:34.943454  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:55:34.943619  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:34.943938  3242 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:55:35.528959  3242 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:35.529157  3242 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:35.529842  3242 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:35.530794  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:55:35.530956  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:55:35.531064  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:35.531307  3242 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:35.652654  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 26: this message will repeat every 5th retry.
I20261118 08:55:36.319967  3285 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:55:36.320152  3285 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:36.320981  3285 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:55:36.321956  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:55:36.322520  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:55:36.322631  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:36.322913  3285 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20261118 08:55:36.791501  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 21: this message will repeat every 5th retry.
W20261118 08:55:36.938033  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 21: this message will repeat every 5th retry.
W20261118 08:55:37.462366  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 31: this message will repeat every 5th retry.
I20261118 08:55:38.354204  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:38.354455  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:55:39.781598  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 31: this message will repeat every 5th retry.
W20261118 08:55:39.964975  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 31: this message will repeat every 5th retry.
W20261118 08:55:41.245625  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 31: this message will repeat every 5th retry.
I20261118 08:55:42.528915  3333 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:42.529117  3333 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:42.530346  3333 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:42.531229  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:55:42.532398  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:55:42.532545  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:55:42.532692  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:42.532941  3333 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:55:42.811623  3332 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:42.811825  3332 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:42.812507  3332 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:42.813449  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:55:42.813611  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:55:42.813727  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:42.813973  3332 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:43.195899  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 31: this message will repeat every 5th retry.
I20261118 08:55:43.355970  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:43.356166  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:55:45.480193  3332 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:45.480386  3332 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:45.481169  3332 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:45.481971  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:55:45.482698  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:55:45.482836  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:45.483126  3332 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:45.683813  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 31: this message will repeat every 5th retry.
W20261118 08:55:46.574167  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 26: this message will repeat every 5th retry.
W20261118 08:55:47.035578  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 26: this message will repeat every 5th retry.
W20261118 08:55:47.488520  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 36: this message will repeat every 5th retry.
I20261118 08:55:48.357723  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:48.357944  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:55:48.801263  3386 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:55:48.801452  3386 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:48.802232  3386 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:55:48.803275  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:55:48.803471  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:55:48.803624  1012 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:48.803907  3386 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:55:49.107906  3252 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:49.108129  3252 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:49.109066  3252 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:49.109977  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:55:49.110144  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:55:49.110285  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:49.110561  3252 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:49.359207  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 36: this message will repeat every 5th retry.
W20261118 08:55:49.925786  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 36: this message will repeat every 5th retry.
W20261118 08:55:51.284401  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 36: this message will repeat every 5th retry.
I20261118 08:55:51.857213  3182 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:51.857421  3182 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:55:51.858119  3182 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:55:51.859035  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:55:51.859642  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:55:51.859743  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:51.860023  3182 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:53.033164  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 36: this message will repeat every 5th retry.
I20261118 08:55:53.359467  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:53.359680  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:55:53.612264  3252 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:55:53.612531  3252 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:55:53.613265  3252 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:55:53.614245  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:55:53.614434  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:55:53.614563  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:55:53.615234  3252 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:55:55.629081  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 36: this message will repeat every 5th retry.
W20261118 08:55:56.517138  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 31: this message will repeat every 5th retry.
W20261118 08:55:56.662532  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 31: this message will repeat every 5th retry.
W20261118 08:55:58.249585  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 41: this message will repeat every 5th retry.
I20261118 08:55:58.361294  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:55:58.361491  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:55:59.317359  3479 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:55:59.317552  3479 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:55:59.318473  3479 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:55:59.319428  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:55:59.320870  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:55:59.321014  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:55:59.321136  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:55:59.321372  3479 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:56:00.030388  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 41: this message will repeat every 5th retry.
W20261118 08:56:00.670131  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 41: this message will repeat every 5th retry.
W20261118 08:56:01.363981  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 41: this message will repeat every 5th retry.
I20261118 08:56:02.567337  3512 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:56:02.567566  3512 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:56:02.568261  3512 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:56:02.570469  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:56:02.570636  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:56:02.570753  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:56:02.570984  3512 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:56:03.363133  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:03.363415  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:03.619737  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 41: this message will repeat every 5th retry.
I20261118 08:56:04.696527  3468 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:04.696734  3468 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:04.697592  3468 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:04.698611  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:56:04.698777  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:56:04.698912  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:04.699177  3468 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:56:05.223538  3469 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:05.223729  3469 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:05.224434  3469 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:05.225507  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:56:05.226583  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:56:05.226745  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:05.227041  3507 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:05.329495  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 41: this message will repeat every 5th retry.
W20261118 08:56:06.033017  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 36: this message will repeat every 5th retry.
W20261118 08:56:07.077019  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 36: this message will repeat every 5th retry.
I20261118 08:56:07.303799  3556 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:56:07.304013  3556 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:56:07.304716  3556 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:56:07.305799  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 08:56:07.306924  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:56:07.307057  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:56:07.307192  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:56:07.307471  3556 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:56:07.475507  3554 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:56:07.476145  3554 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:07.477133  3554 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:56:07.478021  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:56:07.478178  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:56:07.478279  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:07.478523  3554 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:56:08.364980  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:08.365206  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:08.377190  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 46: this message will repeat every 5th retry.
I20261118 08:56:09.102692  3551 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:09.102882  3551 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:09.103664  3551 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:09.104684  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:56:09.104856  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:56:09.104946  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:09.105177  3551 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:56:09.575156  3551 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:09.575352  3551 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:56:09.575985  3551 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:56:09.576920  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:56:09.577099  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:56:09.577193  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:09.577422  3551 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:10.280453  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 46: this message will repeat every 5th retry.
W20261118 08:56:10.542582  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 46: this message will repeat every 5th retry.
W20261118 08:56:10.976244  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 46: this message will repeat every 5th retry.
I20261118 08:56:13.107116  3468 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:13.107352  3468 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:13.109134  3468 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:13.109826  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:56:13.109995  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:56:13.110143  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:13.110420  3468 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:56:13.366639  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:13.366827  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:13.676385  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 46: this message will repeat every 5th retry.
I20261118 08:56:15.253789  3619 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:56:15.254024  3619 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:56:15.254783  3619 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:56:15.255825  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:56:15.257071  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:56:15.257243  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:56:15.257349  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:56:15.257596  3619 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:56:15.387138  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 46: this message will repeat every 5th retry.
W20261118 08:56:15.668114  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 41: this message will repeat every 5th retry.
W20261118 08:56:17.322515  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 41: this message will repeat every 5th retry.
I20261118 08:56:17.717067  3468 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:17.717267  3468 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:56:17.717990  3468 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:56:17.718888  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:56:17.719050  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:56:17.719187  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:17.719466  3468 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:56:18.369143  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:18.369321  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:18.986331  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 51: this message will repeat every 5th retry.
W20261118 08:56:20.174638  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 51: this message will repeat every 5th retry.
W20261118 08:56:20.454113  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 51: this message will repeat every 5th retry.
I20261118 08:56:21.132073  3551 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:21.132306  3551 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:21.133185  3551 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:21.133998  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:56:21.134179  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:56:21.134320  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:21.134579  3551 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:21.345757  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 51: this message will repeat every 5th retry.
I20261118 08:56:23.370868  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:23.371068  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:24.701244  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 51: this message will repeat every 5th retry.
W20261118 08:56:24.821265  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 51: this message will repeat every 5th retry.
W20261118 08:56:25.555078  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 46: this message will repeat every 5th retry.
W20261118 08:56:25.914522  1256 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (765 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:56:25.994333   983 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (266 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:56:26.147709  1121 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (312 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:56:26.618682  3708 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:56:26.618866  3708 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:26.619747  3708 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:56:26.620697  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:56:26.620852  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:56:26.621023  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:26.621294  3708 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20261118 08:56:27.159123  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 46: this message will repeat every 5th retry.
I20261118 08:56:28.372611  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:28.372826  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:29.008963  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 56: this message will repeat every 5th retry.
I20261118 08:56:29.925711  3748 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:29.925987  3748 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:29.926647  3748 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:29.927673  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:56:29.927856  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:56:29.927985  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:29.928287  3748 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:29.930732  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 56: this message will repeat every 5th retry.
I20261118 08:56:30.124997  3754 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:56:30.125273  3754 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:56:30.126106  3754 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:56:30.127000  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:56:30.128423  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:56:30.128595  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:56:30.128767  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:56:30.129030  3754 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:56:30.750974  3748 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:30.751205  3748 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:30.751905  3748 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:30.752823  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:56:30.753000  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:56:30.753130  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:30.753372  3748 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:30.894093  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 56: this message will repeat every 5th retry.
W20261118 08:56:30.950435  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 56: this message will repeat every 5th retry.
I20261118 08:56:30.957609  3764 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:56:30.957772  3764 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:30.958381  3764 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:56:30.959302  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:56:30.959477  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:56:30.959620  1012 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:30.959905  3764 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:56:33.374001  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:33.374167  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:33.728518  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 56: this message will repeat every 5th retry.
I20261118 08:56:34.067198  3748 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:34.067408  3748 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:56:34.068578  3748 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:56:34.069060  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:56:34.069252  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:56:34.069344  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:34.069576  3748 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:35.507803  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 56: this message will repeat every 5th retry.
I20261118 08:56:35.747218  3748 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:35.747476  3748 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:35.748227  3748 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:35.749169  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:56:35.749332  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:56:35.749476  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:35.749755  3748 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:35.853072  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 51: this message will repeat every 5th retry.
W20261118 08:56:36.423677  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 51: this message will repeat every 5th retry.
I20261118 08:56:38.376693  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:38.376899  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:39.576707  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 61: this message will repeat every 5th retry.
W20261118 08:56:40.147802  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 61: this message will repeat every 5th retry.
W20261118 08:56:40.526678  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 61: this message will repeat every 5th retry.
W20261118 08:56:40.870857  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 61: this message will repeat every 5th retry.
I20261118 08:56:42.543140  3749 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:42.543349  3749 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:42.544258  3749 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:42.545132  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:56:42.545733  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:56:42.545837  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:42.546088  3749 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:43.245555  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 61: this message will repeat every 5th retry.
I20261118 08:56:43.348345  3813 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:43.348552  3813 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:56:43.349749  3813 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:56:43.350172  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:56:43.350325  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:56:43.350476  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:43.350739  3813 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:56:43.378424  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:43.378635  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:45.004976  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 56: this message will repeat every 5th retry.
W20261118 08:56:45.552264  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 61: this message will repeat every 5th retry.
I20261118 08:56:46.783135  3886 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:56:46.783317  3886 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:46.784143  3886 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:56:46.785048  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:56:46.785250  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:56:46.785362  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:46.785617  3886 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20261118 08:56:46.916731  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 56: this message will repeat every 5th retry.
I20261118 08:56:48.380386  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:48.380594  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:56:48.998595  3906 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:56:48.998813  3906 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:56:48.999770  3906 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:56:49.004200  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:56:49.005437  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:56:49.005641  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:56:49.005762  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:56:49.006021  3906 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:56:49.719944  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 66: this message will repeat every 5th retry.
I20261118 08:56:49.774554  3918 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:49.774799  3918 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:56:49.775539  3918 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:56:49.776458  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:56:49.776613  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:56:49.776749  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:49.776984  3918 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:50.391311  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 66: this message will repeat every 5th retry.
I20261118 08:56:50.430733  3918 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:50.430938  3918 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:50.431639  3918 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:50.432526  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:56:50.432701  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:56:50.432807  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:50.433034  3918 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:56:50.691210  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 66: this message will repeat every 5th retry.
W20261118 08:56:51.021389  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 66: this message will repeat every 5th retry.
I20261118 08:56:52.601811  2644 maintenance_manager.cc:419] P 857a577b88ba46cc846ecd1a25a722ca: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033355
I20261118 08:56:52.662731  2643 maintenance_manager.cc:643] P 857a577b88ba46cc846ecd1a25a722ca: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.060s	user 0.022s	sys 0.004s Metrics: {"bytes_written":16764,"cfile_init":1,"dirs.queue_time_us":889,"dirs.run_cpu_time_us":213,"dirs.run_wall_time_us":881,"drs_written":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4,"lbm_write_time_us":695,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":38,"thread_start_us":942,"threads_started":2,"wal-append.queue_time_us":330}
I20261118 08:56:52.664049  2644 maintenance_manager.cc:419] P 857a577b88ba46cc846ecd1a25a722ca: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1964 bytes on disk
I20261118 08:56:52.664755  2643 maintenance_manager.cc:643] P 857a577b88ba46cc846ecd1a25a722ca: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4}
I20261118 08:56:53.382496  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:53.382723  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:53.763945  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 66: this message will repeat every 5th retry.
W20261118 08:56:54.763553  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 61: this message will repeat every 5th retry.
W20261118 08:56:54.921329  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 66: this message will repeat every 5th retry.
W20261118 08:56:57.439637  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 61: this message will repeat every 5th retry.
I20261118 08:56:57.449100  3918 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:57.449283  3918 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:57.449903  3918 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:57.450910  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:56:57.451118  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:56:57.451258  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:57.452208  3813 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:56:58.224413  3918 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:56:58.224625  3918 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:56:58.225358  3918 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:56:58.226357  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:56:58.226532  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:56:58.226634  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:56:58.226876  3918 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:56:58.384636  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:56:58.384876  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:56:58.816382  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 71: this message will repeat every 5th retry.
W20261118 08:56:59.408268  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 71: this message will repeat every 5th retry.
W20261118 08:57:00.818118  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 71: this message will repeat every 5th retry.
W20261118 08:57:00.913568  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 71: this message will repeat every 5th retry.
I20261118 08:57:01.579334  4012 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:57:01.579571  4012 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:01.580274  4012 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:57:01.581115  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:57:01.581290  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:57:01.581383  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:01.581595  4012 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:57:02.811450  4029 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:57:02.811668  4029 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:57:02.812465  4029 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:57:02.813709  2639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:57:02.815439  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:57:02.815616  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:57:02.815765  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:57:02.816033  4029 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:57:03.386439  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:03.386622  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:03.730224  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 71: this message will repeat every 5th retry.
W20261118 08:57:04.846489  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 71: this message will repeat every 5th retry.
W20261118 08:57:05.071432  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 66: this message will repeat every 5th retry.
I20261118 08:57:06.426401  4049 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:06.426612  4049 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:57:06.427289  4049 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:57:06.428308  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:57:06.428484  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:57:06.428607  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:06.428861  4049 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:57:07.287979  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 66: this message will repeat every 5th retry.
I20261118 08:57:07.598213  4049 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:07.598405  4049 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:07.598994  4049 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:07.599918  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:57:07.600090  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:57:07.600194  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:07.600435  4049 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:57:07.792465  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 76: this message will repeat every 5th retry.
I20261118 08:57:08.388123  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:08.388317  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:08.585072  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 76: this message will repeat every 5th retry.
W20261118 08:57:10.655377  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 76: this message will repeat every 5th retry.
W20261118 08:57:10.749447  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 76: this message will repeat every 5th retry.
I20261118 08:57:13.389655  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:13.389851  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:13.496822  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 76: this message will repeat every 5th retry.
I20261118 08:57:13.831357  4050 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:13.831593  4050 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:57:13.832554  4050 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:57:13.833523  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:57:13.833685  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:57:13.833845  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:13.834079  4050 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:57:14.454798  3994 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:14.455018  3994 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:14.455883  3994 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:14.456077  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 76: this message will repeat every 5th retry.
W20261118 08:57:14.457000  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:57:14.457168  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:57:14.457310  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:14.457564  3994 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:57:14.797785  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 71: this message will repeat every 5th retry.
W20261118 08:57:16.898777  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 71: this message will repeat every 5th retry.
W20261118 08:57:17.596490  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 81: this message will repeat every 5th retry.
I20261118 08:57:18.391254  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:18.391487  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:18.687974  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 81: this message will repeat every 5th retry.
I20261118 08:57:19.389554  4231 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:19.389760  4231 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:19.390467  4231 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:19.391196  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:57:19.391350  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:57:19.391530  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:19.391784  4231 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:57:20.579126  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 81: this message will repeat every 5th retry.
W20261118 08:57:21.303167  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 81: this message will repeat every 5th retry.
I20261118 08:57:21.365702  4264 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:57:21.365886  4264 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:21.366784  4264 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:57:21.367770  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:57:21.367939  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:57:21.368062  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:21.368319  4264 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:57:21.828130  4273 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:57:21.828332  4273 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:57:21.829241  4273 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:57:21.830199  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:57:21.831758  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:57:21.831933  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:57:21.832042  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:57:21.832307  4273 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:57:22.138616  4231 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:22.138810  4231 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:22.139652  4231 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:22.140278  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:57:22.140488  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:57:22.140599  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:22.140807  4231 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:57:23.392838  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:23.393049  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:23.609048  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 81: this message will repeat every 5th retry.
W20261118 08:57:23.847486  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 81: this message will repeat every 5th retry.
I20261118 08:57:24.438290  4299 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:24.438506  4299 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:24.439153  4299 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:24.440070  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:57:24.440315  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:57:24.440454  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:24.441771  4299 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:57:25.136732  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 76: this message will repeat every 5th retry.
I20261118 08:57:25.658556  4313 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:25.658771  4313 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:57:25.659446  4313 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:57:25.660317  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:57:25.660490  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:57:25.660580  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:25.660792  4313 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:57:25.776654  4334 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:57:25.776886  4334 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:57:25.777635  4334 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:57:25.780184  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:57:25.780369  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:57:25.780481  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:57:25.780716  4334 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:57:25.946666  1256 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (1203 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:57:26.140319   983 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (386 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:57:26.180120  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 76: this message will repeat every 5th retry.
W20261118 08:57:26.488530  1122 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42341 (372 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42341: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:57:27.067734  4354 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:57:27.067942  4354 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:57:27.069057  4354 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:57:27.069917  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 08:57:27.071238  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:57:27.071425  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:57:27.071551  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:57:27.071816  4354 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:57:27.310225  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 86: this message will repeat every 5th retry.
W20261118 08:57:28.060025  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 86: this message will repeat every 5th retry.
I20261118 08:57:28.394341  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:28.394522  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:57:28.864213  4370 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:57:28.864427  4370 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:28.865150  4370 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:57:28.866175  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:57:28.866361  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:57:28.866454  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:28.866681  4370 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20261118 08:57:29.992378  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 86: this message will repeat every 5th retry.
W20261118 08:57:30.968719  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 86: this message will repeat every 5th retry.
I20261118 08:57:32.826208  4314 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:32.826406  4314 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:32.827298  4314 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:32.828336  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:57:32.828920  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:57:32.829025  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:32.829272  4314 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:57:33.003051  4314 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:33.003265  4314 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:33.004040  4314 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:33.004946  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:57:33.005134  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:57:33.005229  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:33.005458  4314 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:57:33.395777  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:33.395953  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:34.058838  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 86: this message will repeat every 5th retry.
W20261118 08:57:34.159585  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 86: this message will repeat every 5th retry.
W20261118 08:57:35.202608  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 81: this message will repeat every 5th retry.
W20261118 08:57:35.974812  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 81: this message will repeat every 5th retry.
W20261118 08:57:36.735183  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 91: this message will repeat every 5th retry.
W20261118 08:57:38.089156  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 91: this message will repeat every 5th retry.
I20261118 08:57:38.375211  4314 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:38.375442  4314 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:57:38.376134  4314 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:57:38.377053  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:57:38.377205  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:57:38.377357  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:38.377610  4314 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:57:38.391023  4314 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:38.391206  4314 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:38.391891  4314 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:38.392889  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:57:38.393100  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:57:38.393255  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:38.393486  4314 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:57:38.397275  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:38.397423  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:40.281062  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 91: this message will repeat every 5th retry.
I20261118 08:57:41.064025  4313 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:41.064213  4313 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:41.064800  4313 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:41.065735  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:57:41.065914  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:57:41.066059  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:41.066287  4313 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:57:41.566818  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 91: this message will repeat every 5th retry.
I20261118 08:57:43.398764  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:43.398944  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:43.972656  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 91: this message will repeat every 5th retry.
W20261118 08:57:44.146862  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 91: this message will repeat every 5th retry.
I20261118 08:57:44.376641  4553 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:57:44.376856  4553 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:57:44.377750  4553 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:57:44.378757  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:57:44.380162  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:57:44.380307  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:57:44.380415  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:57:44.380642  4553 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:57:44.890688  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 86: this message will repeat every 5th retry.
I20261118 08:57:46.291494  4583 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:57:46.291733  4583 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:46.292814  4583 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:57:46.293938  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:57:46.294128  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:57:46.294243  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:46.294536  4583 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20261118 08:57:46.431417  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 96: this message will repeat every 5th retry.
W20261118 08:57:46.655726  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 86: this message will repeat every 5th retry.
W20261118 08:57:48.297097  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 96: this message will repeat every 5th retry.
I20261118 08:57:48.400592  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:48.400816  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:57:48.736032  4540 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:48.736230  4540 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:57:48.737087  4540 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:57:48.738003  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:57:48.738160  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:57:48.738265  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:48.738526  4540 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:57:50.783576  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 96: this message will repeat every 5th retry.
W20261118 08:57:51.245422  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 96: this message will repeat every 5th retry.
I20261118 08:57:51.670264  4657 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:57:51.670480  4657 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:51.671205  4657 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:57:51.672215  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:57:51.672395  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:57:51.672531  1012 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:51.673242  4657 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:57:52.485152  4681 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:57:52.485369  4681 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:57:52.486109  4681 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:57:52.487103  2639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:57:52.488272  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:57:52.488449  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:57:52.488554  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:57:52.488758  4681 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:57:53.160959  4678 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:53.161157  4678 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:53.161870  4678 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:53.162869  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:57:53.163060  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:57:53.163156  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:53.163399  4678 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:57:53.402053  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:53.402244  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:57:53.524145  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 96: this message will repeat every 5th retry.
W20261118 08:57:54.312764  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 96: this message will repeat every 5th retry.
W20261118 08:57:55.071861  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 91: this message will repeat every 5th retry.
W20261118 08:57:56.021706  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 91: this message will repeat every 5th retry.
W20261118 08:57:56.566620  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 101: this message will repeat every 5th retry.
I20261118 08:57:57.044891  4742 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:57:57.045120  4742 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:57:57.045958  4742 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:57:57.047039  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:57:57.047231  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:57:57.047358  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:57:57.047664  4742 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:57:58.063182  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 101: this message will repeat every 5th retry.
I20261118 08:57:58.404238  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:57:58.404412  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:58:00.476320  4741 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:00.476521  4741 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:00.477169  4741 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:00.478080  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:58:00.478230  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:58:00.478367  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:00.478607  4741 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:00.626058  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 101: this message will repeat every 5th retry.
W20261118 08:58:01.224153  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 101: this message will repeat every 5th retry.
I20261118 08:58:01.458575  4832 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:58:01.458774  4832 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:01.459609  4832 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:58:01.460556  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:58:01.460734  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:58:01.460877  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:01.461124  4832 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:58:01.837862  4838 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:01.838052  4838 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:01.838773  4838 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:01.839820  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:58:01.840929  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:58:01.841061  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:58:01.841203  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:01.841441  4838 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:58:03.405916  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:03.406118  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:03.720842  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 101: this message will repeat every 5th retry.
W20261118 08:58:03.870702  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 101: this message will repeat every 5th retry.
W20261118 08:58:04.514531  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 96: this message will repeat every 5th retry.
W20261118 08:58:05.857724  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 96: this message will repeat every 5th retry.
I20261118 08:58:05.903087  4896 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:05.903311  4896 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:58:05.904120  4896 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:58:05.904906  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:58:05.905090  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:58:05.905206  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:05.905431  4896 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:58:07.272198  4896 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:07.272421  4896 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:07.273038  4896 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:07.273949  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:58:07.274128  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:58:07.274224  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:07.274467  4896 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:58:07.442584  4931 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:58:07.442770  4931 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:07.443519  4931 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:58:07.444458  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:58:07.444646  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:58:07.444777  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:07.445013  4931 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20261118 08:58:07.616060  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 106: this message will repeat every 5th retry.
W20261118 08:58:07.771492  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 106: this message will repeat every 5th retry.
I20261118 08:58:08.407959  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:08.408164  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:58:09.211404  4932 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:09.211655  4932 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:09.212359  4932 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:09.213320  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:58:09.213482  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:58:09.213601  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:09.213835  4932 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:58:10.523654  4981 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:10.523870  4981 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:10.524664  4981 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:10.526247  2639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:58:10.527400  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:58:10.527592  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:58:10.527729  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:10.528002  4981 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:58:10.605207  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 106: this message will repeat every 5th retry.
W20261118 08:58:10.729740  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 106: this message will repeat every 5th retry.
I20261118 08:58:11.769147  5002 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:11.769343  5002 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:11.770032  5002 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:11.772464  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:58:11.772655  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:58:11.772756  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:11.772970  5002 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:58:13.239437  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 106: this message will repeat every 5th retry.
I20261118 08:58:13.409818  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:13.410010  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:58:13.869660  5040 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:58:13.869935  5040 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
W20261118 08:58:13.871131  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 106: this message will repeat every 5th retry.
I20261118 08:58:13.871647  5040 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:58:13.872478  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:58:13.872601  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:58:13.872759  1012 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:13.873013  5040 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:58:13.914664  4933 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:13.914865  4933 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:13.915640  4933 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:13.916586  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:58:13.916769  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:58:13.916903  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:13.917186  4933 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:14.086374  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 101: this message will repeat every 5th retry.
I20261118 08:58:14.991521  5056 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:14.991725  5056 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:14.992527  5056 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:14.997365  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:58:14.997550  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:58:14.998112  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:14.998428  5056 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:58:15.795671  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 101: this message will repeat every 5th retry.
I20261118 08:58:15.996923  4933 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:15.997119  4933 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:58:15.997906  4933 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:58:15.998978  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:58:15.999141  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:58:15.999282  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:15.999564  4933 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:17.211984  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 111: this message will repeat every 5th retry.
W20261118 08:58:17.652159  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 111: this message will repeat every 5th retry.
I20261118 08:58:17.998585  5106 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:17.998780  5106 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:17.999498  5106 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:18.000538  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20261118 08:58:18.002205  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:58:18.002370  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:58:18.002485  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:18.002753  5106 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:58:18.411355  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:18.411602  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:20.220495  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 111: this message will repeat every 5th retry.
W20261118 08:58:21.214172  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 111: this message will repeat every 5th retry.
W20261118 08:58:23.025146  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 111: this message will repeat every 5th retry.
I20261118 08:58:23.413147  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:23.413333  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:23.712766  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 111: this message will repeat every 5th retry.
W20261118 08:58:24.020530  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 106: this message will repeat every 5th retry.
I20261118 08:58:25.256313  5216 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:25.256512  5216 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:25.257220  5216 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:25.258222  2639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:58:25.258247  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:58:25.259526  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:58:25.259686  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:58:25.259807  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:25.260017  5216 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:58:26.000677  1256 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (1630 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:58:26.029651  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 106: this message will repeat every 5th retry.
W20261118 08:58:26.308990  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 116: this message will repeat every 5th retry.
W20261118 08:58:26.317118   983 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (506 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:58:26.817726  1122 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42341 (521 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42341: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:58:26.827726  4933 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:26.827901  4933 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:26.828666  4933 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:26.829648  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:58:26.829793  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:58:26.829875  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:26.830077  4933 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:27.911135  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 116: this message will repeat every 5th retry.
I20261118 08:58:28.414810  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:28.415010  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:30.234118  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 116: this message will repeat every 5th retry.
I20261118 08:58:31.916823  4896 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:31.917011  4896 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:58:31.917696  4896 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:58:31.918629  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:58:31.918766  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:58:31.918849  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:31.919044  4896 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:32.000252  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 116: this message will repeat every 5th retry.
I20261118 08:58:32.294178  4896 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:32.294350  4896 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:32.294960  4896 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:32.295940  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:58:32.296077  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:58:32.296203  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:32.296473  4896 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:58:32.575585  4896 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:32.575778  4896 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:32.576421  4896 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:32.577317  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:58:32.577447  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:58:32.577556  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:32.577790  4896 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:32.589865  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 116: this message will repeat every 5th retry.
I20261118 08:58:32.875303  5331 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:32.875540  5331 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:32.876706  5331 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:32.877676  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:58:32.878756  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:58:32.878897  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:58:32.879011  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:32.879268  5331 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:58:33.396426  5341 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:58:33.396607  5341 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:33.397483  5341 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:58:33.398399  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:58:33.398571  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:58:33.398659  1012 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:33.398870  5341 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:58:33.416570  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:33.416781  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:33.618379  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 116: this message will repeat every 5th retry.
W20261118 08:58:34.010331  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 111: this message will repeat every 5th retry.
W20261118 08:58:35.366449  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 111: this message will repeat every 5th retry.
W20261118 08:58:36.684237  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 121: this message will repeat every 5th retry.
W20261118 08:58:37.597950  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 121: this message will repeat every 5th retry.
I20261118 08:58:38.418232  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:38.418443  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:39.820638  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 121: this message will repeat every 5th retry.
W20261118 08:58:41.892690  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 121: this message will repeat every 5th retry.
W20261118 08:58:42.113730  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 121: this message will repeat every 5th retry.
W20261118 08:58:43.080662  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 121: this message will repeat every 5th retry.
I20261118 08:58:43.419965  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:43.420174  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:43.757220  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 116: this message will repeat every 5th retry.
W20261118 08:58:45.454900  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 116: this message will repeat every 5th retry.
I20261118 08:58:46.546798  5502 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:46.547006  5502 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:58:46.547993  5502 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:58:46.548974  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:58:46.549110  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:58:46.549193  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:46.549408  5502 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:46.639245  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 126: this message will repeat every 5th retry.
W20261118 08:58:47.498363  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 126: this message will repeat every 5th retry.
I20261118 08:58:48.422314  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:48.422570  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:58:49.046934  5562 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:49.047134  5562 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:49.047889  5562 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:49.048929  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:58:49.049073  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:58:49.049204  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:49.049477  5562 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:49.315145  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 126: this message will repeat every 5th retry.
I20261118 08:58:49.461073  5572 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:58:49.461298  5572 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:49.462234  5572 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:58:49.463202  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:58:49.463716  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:58:49.463868  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:49.464308  5572 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:58:49.506397  5583 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:49.506636  5583 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:49.507725  5583 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:49.508646  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:58:49.510548  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:58:49.510771  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:58:49.510913  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:49.511158  5583 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:58:51.696516  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 126: this message will repeat every 5th retry.
I20261118 08:58:52.517630  5623 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:58:52.517808  5623 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:52.518447  5623 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:58:52.519479  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:58:52.519641  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:58:52.519788  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:52.520040  5623 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:58:52.539234  5562 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:58:52.539469  5562 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:58:52.540021  5562 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:58:52.540970  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:58:52.541103  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:58:52.541220  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:58:52.541451  5562 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:58:52.667519  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 126: this message will repeat every 5th retry.
W20261118 08:58:53.138473  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 126: this message will repeat every 5th retry.
I20261118 08:58:53.424280  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:53.424507  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:53.622447  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 121: this message will repeat every 5th retry.
W20261118 08:58:55.350610  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 121: this message will repeat every 5th retry.
W20261118 08:58:57.431314  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 131: this message will repeat every 5th retry.
W20261118 08:58:57.928843  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 131: this message will repeat every 5th retry.
I20261118 08:58:58.426076  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:58:58.426276  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:58:58.868916  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 131: this message will repeat every 5th retry.
I20261118 08:58:58.879704  5718 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:58:58.879882  5718 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:58:58.880546  5718 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:58:58.881071  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:58:58.882385  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:58:58.882579  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:58:58.882686  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:58:58.882895  5718 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:59:01.567564  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 131: this message will repeat every 5th retry.
I20261118 08:59:02.426470  5773 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:02.426652  5773 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:02.427489  5773 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:02.428565  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:59:02.428753  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:59:02.428864  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:02.429086  5773 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:02.976280  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 131: this message will repeat every 5th retry.
I20261118 08:59:03.427755  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:03.427986  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:03.493309  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 126: this message will repeat every 5th retry.
W20261118 08:59:03.526296  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 131: this message will repeat every 5th retry.
I20261118 08:59:03.842165  5773 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:03.842361  5773 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:59:03.843048  5773 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:59:03.844028  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:59:03.844183  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:59:03.844280  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:03.844485  5774 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:59:04.585103  5814 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:59:04.585306  5814 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:59:04.586090  5814 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:59:04.587074  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:59:04.588922  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:59:04.589155  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:59:04.589300  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:59:04.589541  5814 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:59:04.966760  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 126: this message will repeat every 5th retry.
W20261118 08:59:07.393180  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 136: this message will repeat every 5th retry.
W20261118 08:59:08.357748  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 136: this message will repeat every 5th retry.
I20261118 08:59:08.429478  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:08.429675  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 08:59:08.546449  5773 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:08.546638  5773 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:08.547279  5773 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:08.548290  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:59:08.548434  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:59:08.548518  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:08.548717  5773 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:59:09.048475  5881 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:59:09.048702  5881 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:59:09.049474  5881 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:59:09.052294  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:59:09.052461  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:59:09.052603  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:59:09.052891  5881 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:59:09.093204  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 136: this message will repeat every 5th retry.
I20261118 08:59:09.637414  5885 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:59:09.637576  5885 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:09.638411  5885 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:59:09.639358  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:59:09.639535  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:59:09.639660  1012 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:09.639889  5885 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:59:10.235421  5903 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:59:10.235605  5903 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:59:10.236227  5903 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:59:10.237162  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 08:59:10.238152  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:59:10.238327  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:59:10.238447  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:59:10.238684  5903 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:59:11.955576  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 136: this message will repeat every 5th retry.
I20261118 08:59:12.268244  5822 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:12.268438  5822 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:12.269086  5822 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:12.270097  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:59:12.270241  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:59:12.270324  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:12.270519  5822 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:12.760205  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 136: this message will repeat every 5th retry.
I20261118 08:59:13.431155  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:13.431352  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:13.441566  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 131: this message will repeat every 5th retry.
I20261118 08:59:13.466651  5822 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:13.466894  5822 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:59:13.467643  5822 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:59:13.468776  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:59:13.468936  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:59:13.469044  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:13.469295  5822 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:13.479260  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 136: this message will repeat every 5th retry.
I20261118 08:59:14.239529  5774 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:14.239713  5774 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:14.240415  5774 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:14.241333  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:59:14.241468  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:59:14.241549  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:14.241740  5774 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:15.662703  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 131: this message will repeat every 5th retry.
W20261118 08:59:16.649622  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 141: this message will repeat every 5th retry.
I20261118 08:59:18.432866  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:18.433055  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:18.799332  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 141: this message will repeat every 5th retry.
W20261118 08:59:18.961671  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 141: this message will repeat every 5th retry.
I20261118 08:59:19.937129  5773 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:19.937314  5773 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:19.938084  5773 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:19.938966  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:59:19.939101  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:59:19.939180  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:19.939373  5773 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:22.930121  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 141: this message will repeat every 5th retry.
W20261118 08:59:23.289916  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 141: this message will repeat every 5th retry.
I20261118 08:59:23.434757  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:23.434917  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:23.452252  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 141: this message will repeat every 5th retry.
W20261118 08:59:23.612715  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 136: this message will repeat every 5th retry.
I20261118 08:59:24.716600  6104 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:24.716773  6104 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:59:24.717419  6104 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:59:24.718353  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:59:24.718483  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:59:24.718606  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:24.718817  6104 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:25.426488  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 136: this message will repeat every 5th retry.
W20261118 08:59:26.145167  1256 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (2053 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:59:26.207154  6145 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:26.207489  6145 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:26.208187  6145 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:26.208881  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:59:26.209022  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:59:26.209111  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:26.209309  6145 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:26.525099   983 heartbeater.cc:648] Failed to heartbeat to 127.0.72.188:41259 (626 consecutive failures): Network error: Failed to ping master at 127.0.72.188:41259: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:59:26.589147  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 146: this message will repeat every 5th retry.
W20261118 08:59:26.840677  1122 heartbeater.cc:648] Failed to heartbeat to 127.0.72.190:42341 (668 consecutive failures): Network error: Failed to ping master at 127.0.72.190:42341: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:59:27.521755  6164 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:59:27.521924  6164 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:27.522910  6164 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:59:27.523799  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:59:27.523941  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:59:27.524058  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:27.524503  6164 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:59:27.950042  6171 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:59:27.950233  6171 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:59:27.951045  6171 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:59:27.951957  2640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:59:27.953095  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
W20261118 08:59:27.953215  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
I20261118 08:59:27.953303  2639 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:59:27.953549  6171 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 08:59:28.436081  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:28.436249  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:28.542209  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 146: this message will repeat every 5th retry.
W20261118 08:59:28.685345  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 146: this message will repeat every 5th retry.
I20261118 08:59:29.914248  6196 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:29.914433  6196 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:29.915042  6196 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:29.916085  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:59:29.916234  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:59:29.916349  1147 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:29.916597  6196 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:32.528707  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 141: this message will repeat every 5th retry.
W20261118 08:59:32.575800  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 146: this message will repeat every 5th retry.
I20261118 08:59:33.228451  6243 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:33.228667  6243 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:33.229346  6243 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:33.230394  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:59:33.230521  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:59:33.230650  1146 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:33.230899  6243 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:33.243453  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 146: this message will repeat every 5th retry.
W20261118 08:59:33.397771  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 146: this message will repeat every 5th retry.
I20261118 08:59:33.437693  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:33.437855  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:34.672181  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 141: this message will repeat every 5th retry.
W20261118 08:59:36.994298  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 151: this message will repeat every 5th retry.
I20261118 08:59:37.215531  6278 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:37.215731  6278 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:59:37.216572  6278 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:59:37.217518  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:59:37.217672  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:59:37.217787  1147 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:37.218015  6278 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 08:59:38.439198  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:38.439344  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:39.181041  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 151: this message will repeat every 5th retry.
W20261118 08:59:39.234125  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 151: this message will repeat every 5th retry.
I20261118 08:59:41.682808  6394 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:59:41.682978  6394 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:59:41.683753  6394 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:59:41.684640  2639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 08:59:41.685710  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:59:41.685855  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:59:41.685954  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:59:41.686161  6394 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:59:42.234537  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 146: this message will repeat every 5th retry.
W20261118 08:59:42.300614  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 151: this message will repeat every 5th retry.
W20261118 08:59:43.085206  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 151: this message will repeat every 5th retry.
I20261118 08:59:43.440968  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:43.441165  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:44.173602  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 151: this message will repeat every 5th retry.
I20261118 08:59:44.314772  6280 raft_consensus.cc:499] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:44.314996  6280 raft_consensus.cc:521] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:44.315663  6280 leader_election.cc:290] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:44.316654  1146 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
W20261118 08:59:44.316818  1147 leader_election.cc:343] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 48a6e2d96fe442b6a84b4773993b4f54
I20261118 08:59:44.316941  1147 leader_election.cc:304] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:44.317176  6280 raft_consensus.cc:2771] T 48a6e2d96fe442b6a84b4773993b4f54 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:44.949927  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 146: this message will repeat every 5th retry.
I20261118 08:59:45.331307  6448 raft_consensus.cc:499] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 78d4026bc230421ea10b8c68a9ef7871)
I20261118 08:59:45.331562  6448 raft_consensus.cc:521] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78d4026bc230421ea10b8c68a9ef7871" member_type: VOTER last_known_addr { host: "127.0.72.190" port: 42341 } } peers { permanent_uuid: "857a577b88ba46cc846ecd1a25a722ca" member_type: VOTER last_known_addr { host: "127.0.72.189" port: 33939 } } peers { permanent_uuid: "6c7d9f52503f41e4bafe2821e6fd6af1" member_type: VOTER last_known_addr { host: "127.0.72.188" port: 41259 } }
I20261118 08:59:45.332829  6448 leader_election.cc:290] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341), 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259)
W20261118 08:59:45.341427  2639 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7d9f52503f41e4bafe2821e6fd6af1 (127.0.72.188:41259): Network error: Client connection negotiation failed: client connection to 127.0.72.188:41259: connect: Connection refused (error 111)
W20261118 08:59:45.342938  2640 leader_election.cc:336] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 78d4026bc230421ea10b8c68a9ef7871 (127.0.72.190:42341): Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111)
I20261118 08:59:45.343056  2640 leader_election.cc:304] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 857a577b88ba46cc846ecd1a25a722ca; no voters: 6c7d9f52503f41e4bafe2821e6fd6af1, 78d4026bc230421ea10b8c68a9ef7871
I20261118 08:59:45.343267  6448 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 857a577b88ba46cc846ecd1a25a722ca [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20261118 08:59:46.490533  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 156: this message will repeat every 5th retry.
I20261118 08:59:46.903807  6479 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:59:46.903965  6479 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:46.904706  6479 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:59:46.905514  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:59:46.905635  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:59:46.905757  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:46.905967  6479 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:59:48.442613  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:48.442806  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:48.678619  1146 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 156: this message will repeat every 5th retry.
W20261118 08:59:48.863859  1147 consensus_peers.cc:597] T d7f4643e2be049d7b2e3249a40c6b2ac P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7f4643e2be049d7b2e3249a40c6b2ac. This is attempt 156: this message will repeat every 5th retry.
I20261118 08:59:49.659436  6508 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:49.659582  6508 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:49.660073  6508 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:49.660934  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:59:49.661060  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:59:49.661172  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:49.661383  6508 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:52.774825  1147 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 156: this message will repeat every 5th retry.
W20261118 08:59:52.820739  1147 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 151: this message will repeat every 5th retry.
W20261118 08:59:53.098852  1147 consensus_peers.cc:597] T 3e292319f0c44e34b49ae6d579df2ecb P 3934fbc351ee4a99add35742e90dfa68 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e292319f0c44e34b49ae6d579df2ecb. This is attempt 156: this message will repeat every 5th retry.
I20261118 08:59:53.444140  2617 master_runner.cc:187] Error getting master registration for 127.0.72.190:42341: Network error: Client connection negotiation failed: client connection to 127.0.72.190:42341: connect: Connection refused (error 111), 
I20261118 08:59:53.444365  2617 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 08:59:53.546476  1012 consensus_peers.cc:597] T fc173583a9584741b28f2c0203504a3a P 213b5f7a64e74b109a3a9387e5975042 -> Peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Couldn't send request to peer ad2789ff5b54489c84966cff78be4721. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc173583a9584741b28f2c0203504a3a. This is attempt 156: this message will repeat every 5th retry.
I20261118 08:59:54.863211  6508 raft_consensus.cc:499] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:54.863432  6508 raft_consensus.cc:521] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } }
I20261118 08:59:54.864207  6508 leader_election.cc:290] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649), ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287)
W20261118 08:59:54.865281  1147 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
W20261118 08:59:54.865392  1146 leader_election.cc:343] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: 8d92c02e29724e80820a7114d8bbdf7a
I20261118 08:59:54.865527  1146 leader_election.cc:304] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:54.865759  6508 raft_consensus.cc:2771] T 8d92c02e29724e80820a7114d8bbdf7a P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:55.022512  1146 consensus_peers.cc:597] T 4eecc49bfe014d1d86c84e8a4f16a5bf P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4eecc49bfe014d1d86c84e8a4f16a5bf. This is attempt 151: this message will repeat every 5th retry.
I20261118 08:59:55.787971  6508 raft_consensus.cc:499] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad2789ff5b54489c84966cff78be4721)
I20261118 08:59:55.788156  6508 raft_consensus.cc:521] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:55.788782  6508 leader_election.cc:290] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649)
W20261118 08:59:55.789942  1147 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
W20261118 08:59:55.790118  1146 leader_election.cc:343] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Not found: Tablet not found: d92e535f2bf44815b6c003b56dd25b79
I20261118 08:59:55.790246  1146 leader_election.cc:304] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3934fbc351ee4a99add35742e90dfa68; no voters: 213b5f7a64e74b109a3a9387e5975042, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:55.790508  6508 raft_consensus.cc:2771] T d92e535f2bf44815b6c003b56dd25b79 P 3934fbc351ee4a99add35742e90dfa68 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 08:59:56.114533  1146 consensus_peers.cc:597] T 39e7203496e0458687be52c3fadd15ea P 3934fbc351ee4a99add35742e90dfa68 -> Peer 213b5f7a64e74b109a3a9387e5975042 (127.0.72.130:33649): Couldn't send request to peer 213b5f7a64e74b109a3a9387e5975042. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 39e7203496e0458687be52c3fadd15ea. This is attempt 161: this message will repeat every 5th retry.
I20261118 08:59:56.823601  6599 raft_consensus.cc:499] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 08:59:56.823793  6599 raft_consensus.cc:521] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2789ff5b54489c84966cff78be4721" member_type: VOTER last_known_addr { host: "127.0.72.129" port: 39287 } } peers { permanent_uuid: "213b5f7a64e74b109a3a9387e5975042" member_type: VOTER last_known_addr { host: "127.0.72.130" port: 33649 } } peers { permanent_uuid: "3934fbc351ee4a99add35742e90dfa68" member_type: VOTER last_known_addr { host: "127.0.72.131" port: 46447 } }
I20261118 08:59:56.824525  6599 leader_election.cc:290] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287), 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447)
W20261118 08:59:56.825538  1012 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ad2789ff5b54489c84966cff78be4721 (127.0.72.129:39287): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
W20261118 08:59:56.825711  1011 leader_election.cc:343] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3934fbc351ee4a99add35742e90dfa68 (127.0.72.131:46447): Not found: Tablet not found: 380fa8ff3e2f428b81b7567bc7bcefee
I20261118 08:59:56.825819  1011 leader_election.cc:304] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 213b5f7a64e74b109a3a9387e5975042; no voters: 3934fbc351ee4a99add35742e90dfa68, ad2789ff5b54489c84966cff78be4721
I20261118 08:59:56.826076  6599 raft_consensus.cc:2771] T 380fa8ff3e2f428b81b7567bc7bcefee P 213b5f7a64e74b109a3a9387e5975042 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 08:59:56.949539  2644 maintenance_manager.cc:419] P 857a577b88ba46cc846ecd1a25a722ca: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 1964 bytes on disk
I20261118 08:59:56.954752  2643 maintenance_manager.cc:643] P 857a577b88ba46cc846ecd1a25a722ca: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s	user 0.004s	sys 0.000s Metrics: {"dirs.queue_time_us":340,"thread_start_us":525,"threads_started":2,"wal-append.queue_time_us":421}
W20261118 08:59:57.664301  1264 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
W20261118 08:59:57.664523  1264 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
W20261118 08:59:57.664608  1264 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
W20261118 08:59:57.664665  1264 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
F20261118 08:59:57.675375  1297 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
*** Check failure stack trace: ***
*** Aborted at 1794992397 (unix time) try "date -d @1794992397" if you are using GNU date ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1276 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_903cefd8f2a74c89be4bf41b84d26ba4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
PC: @                0x0 (unknown)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1276 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_903cefd8f2a74c89be4bf41b84d26ba4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681154  1288 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681181  1291 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
*** SIGABRT (@0x3e800000122) received by PID 290 (TID 0x7f0cbcf49700) from PID 290; stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1276 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_903cefd8f2a74c89be4bf41b84d26ba4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681154  1288 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681181  1291 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.682323  1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
    @     0x7f0cdc494980 (unknown) at ??:0
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1276 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_903cefd8f2a74c89be4bf41b84d26ba4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681154  1288 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681181  1291 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.682323  1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683583  1281 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1276 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_903cefd8f2a74c89be4bf41b84d26ba4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681154  1288 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681181  1291 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.682323  1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683583  1281 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683658  1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_52bba9536ae942a59235d8087d61576b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
    @     0x7f0cd5204e87 gsignal at ??:0
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1276 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_903cefd8f2a74c89be4bf41b84d26ba4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681154  1288 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681181  1291 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.682323  1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683583  1281 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683658  1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_52bba9536ae942a59235d8087d61576b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683946  1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1276 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_903cefd8f2a74c89be4bf41b84d26ba4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681154  1288 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681181  1291 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.682323  1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683583  1281 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683658  1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_52bba9536ae942a59235d8087d61576b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683946  1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683988  1285 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
F20261118 08:59:57.675616  1289 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675695  1284 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675706  1283 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675868  1277 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8f6feb81c1374d42bc1e0acb856b75cd on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.675885  1272 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_9dbf0bcdd2ac45978f8b2c56be77c11a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677057  1296 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677203  1273 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_738b3d8aface40768cac1e7b01d47046 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677455  1280 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_3655f04cfdf84b26839c6e76aac6ebbe on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677484  1278 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bfd124a61a904e60bc99bee8716f64cc on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677584  1287 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.677711  1286 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.678719  1275 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_82d1541f550a4d5482cff56810c97a22 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679875  1271 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_0327d29a2f19450b872c54c554d7654e on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1292 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.679920  1276 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_903cefd8f2a74c89be4bf41b84d26ba4 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681154  1288 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.681181  1291 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.682323  1293 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683583  1281 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683658  1279 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_52bba9536ae942a59235d8087d61576b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683946  1282 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.683988  1285 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)F20261118 08:59:57.684332  1274 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_b5863ec6b7e144f58594f23a0634bf0a on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 771) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.0.72.190:42341,127.0.72.189:33939,127.0.72.188:41259, num_attempts: 1)
*** Check failure stack trace: ***
    @     0x7f0cd52067f1 abort at ??:0
    @     0x7f0cd7f53d5d google::LogMessage::Fail() at ??:0
    @     0x7f0cd7f57b23 google::LogMessage::SendToLog() at ??:0
    @     0x7f0cd7f5375c google::LogMessage::Flush() at ??:0
    @     0x7f0cd7f54ee9 google::LogMessageFatal::~LogMessageFatal() at ??:0
    @           0x40b4d5 kudu::MasterStressTest::ReplaceTabletThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:393
    @           0x447484 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
    @           0x44733d std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
    @           0x4472fa _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
    @           0x4472ae std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
    @           0x446fe9 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
    @     0x7f0cd588e6df (unknown) at ??:0
    @     0x7f0cdc4896db start_thread at ??:0
    @     0x7f0cd52e761f clone at ??:0