Diagnosed failure

CatalogManagerConfigurations/MasterStressTest.Test/2: Unrecognized error type. Please see the error log for more information.

Full log

Note: This is test shard 3 of 3.
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CatalogManagerConfigurations/MasterStressTest
[ RUN      ] CatalogManagerConfigurations/MasterStressTest.Test/2
2026-11-18T02:54:41Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-11-18T02:54:41Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task8fD3U4/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task8fD3U4/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:derby:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/metadb;create=true
Metastore Connection Driver :	 org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:	 APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 

 
 
 
 

 
 
 
 


 
 
 

 
 
 


 
 
 
 
 
 
 


 
 
 

 
 
 

 
 

 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 


 
 
 
 
 
 



 
 
 
 
 


 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 


Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/security.properties
2026-11-18 02:54:48: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task8fD3U4/thirdparty/installed/common/opt/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task8fD3U4/thirdparty/installed/common/opt/hadoop/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20261118 02:54:56.114955  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.126:38025
--webserver_interface=127.2.229.126
--webserver_port=0
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 02:54:56.452553  3246 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:54:56.452838  3246 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:54:56.452914  3246 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:54:56.460963  3246 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:54:56.461047  3246 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:54:56.461100  3246 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:54:56.461146  3246 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:54:56.471426  3246 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.126:38025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.229.126
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:54:56.473129  3246 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:54:56.474905  3246 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:54:56.484417  3252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:56.484465  3251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:56.484555  3254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:54:56.485175  3246 server_base.cc:1061] running on GCE node
I20261118 02:54:56.485896  3246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:54:56.487358  3246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:54:56.488569  3246 hybrid_clock.cc:648] HybridClock initialized: now 1794970496488476 us; error 79 us; skew 500 ppm
I20261118 02:54:56.488901  3246 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:54:56.490999  3246 webserver.cc:577] Webserver started at http://127.2.229.126:44955/ using document root <none> and password file <none>
I20261118 02:54:56.491662  3246 fs_manager.cc:362] Metadata directory not provided
I20261118 02:54:56.491770  3246 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:54:56.492172  3246 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 02:54:56.494459  3246 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/instance:
uuid: "26c6b20fa203404a88463ea27bdd3c7d"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:54:56.495209  3246 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal/instance:
uuid: "26c6b20fa203404a88463ea27bdd3c7d"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:54:56.500836  3246 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20261118 02:54:56.504983  3260 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:56.506525  3246 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20261118 02:54:56.506685  3246 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
uuid: "26c6b20fa203404a88463ea27bdd3c7d"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:54:56.506874  3246 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:54:56.532059  3246 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:54:56.533010  3246 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:54:56.533267  3246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:54:56.552716  3246 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.126:38025
I20261118 02:54:56.552734  3311 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.126:38025 every 8 connection(s)
I20261118 02:54:56.554250  3246 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
I20261118 02:54:56.559012  3312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:54:56.561719  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 3246
I20261118 02:54:56.562021  2965 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal/instance
I20261118 02:54:56.565382  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.125:38339
--webserver_interface=127.2.229.125
--webserver_port=0
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 02:54:56.566968  3312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } has no permanent_uuid. Determining permanent_uuid...
I20261118 02:54:56.585426  3312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } has no permanent_uuid. Determining permanent_uuid...
W20261118 02:54:56.586551  3262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:54:56.588035  3312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.125:38339: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:54:56.637395  3312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } attempt: 1
W20261118 02:54:56.639678  3312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.125:38339: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:54:56.707960  3312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } attempt: 2
W20261118 02:54:56.710085  3312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.125:38339: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:54:56.801340  3312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } attempt: 3
W20261118 02:54:56.803478  3312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.125:38339: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:54:56.902096  3313 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:54:56.902383  3313 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:54:56.902487  3313 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:54:56.911231  3313 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:54:56.911329  3313 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:54:56.911386  3313 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:54:56.911448  3313 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:54:56.922199  3313 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.125:38339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.229.125
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:54:56.923800  3313 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:54:56.925546  3313 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:54:56.933743  3321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:56.933895  3322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:56.934798  3324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:54:56.935501  3313 server_base.cc:1061] running on GCE node
I20261118 02:54:56.936355  3313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:54:56.937865  3313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:54:56.939128  3313 hybrid_clock.cc:648] HybridClock initialized: now 1794970496939055 us; error 58 us; skew 500 ppm
I20261118 02:54:56.939474  3313 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:54:56.941597  3313 webserver.cc:577] Webserver started at http://127.2.229.125:39093/ using document root <none> and password file <none>
I20261118 02:54:56.942291  3313 fs_manager.cc:362] Metadata directory not provided
I20261118 02:54:56.942401  3313 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:54:56.942763  3313 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 02:54:56.945137  3313 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/instance:
uuid: "457eb38cba564ecd9f6f11245172260b"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:54:56.945937  3313 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal/instance:
uuid: "457eb38cba564ecd9f6f11245172260b"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:54:56.946704  3312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } attempt: 4
W20261118 02:54:56.948648  3312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.125:38339: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:54:56.951696  3313 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20261118 02:54:56.955060  3330 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:56.956616  3313 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20261118 02:54:56.956769  3313 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
uuid: "457eb38cba564ecd9f6f11245172260b"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:54:56.956960  3313 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:54:56.987924  3313 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:54:56.988865  3313 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:54:56.989114  3313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:54:57.010638  3313 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.125:38339
I20261118 02:54:57.010656  3381 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.125:38339 every 8 connection(s)
I20261118 02:54:57.012235  3313 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
I20261118 02:54:57.014909  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 3313
I20261118 02:54:57.015105  2965 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal/instance
I20261118 02:54:57.016510  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.124:36695
--webserver_interface=127.2.229.124
--webserver_port=0
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 02:54:57.017807  3382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:54:57.024376  3382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } has no permanent_uuid. Determining permanent_uuid...
I20261118 02:54:57.040587  3382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } has no permanent_uuid. Determining permanent_uuid...
I20261118 02:54:57.050233  3382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } has no permanent_uuid. Determining permanent_uuid...
W20261118 02:54:57.051327  3333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
W20261118 02:54:57.052856  3382 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:54:57.102182  3382 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } attempt: 1
W20261118 02:54:57.104190  3382 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:54:57.172462  3382 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } attempt: 2
W20261118 02:54:57.174477  3382 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:54:57.247923  3312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } attempt: 5
I20261118 02:54:57.254782  3312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } has no permanent_uuid. Determining permanent_uuid...
W20261118 02:54:57.256673  3312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:54:57.265739  3382 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } attempt: 3
W20261118 02:54:57.267822  3382 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:54:57.307953  3312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } attempt: 1
W20261118 02:54:57.310200  3312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
W20261118 02:54:57.361761  3383 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:54:57.362092  3383 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:54:57.362171  3383 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:54:57.370635  3383 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:54:57.370723  3383 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:54:57.370775  3383 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:54:57.370821  3383 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:54:57.378453  3312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } attempt: 2
W20261118 02:54:57.380460  3312 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:54:57.381361  3383 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.124:36695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.229.124
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:54:57.382948  3383 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:54:57.384711  3383 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:54:57.393571  3391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:57.393539  3392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:57.393714  3394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:54:57.394506  3383 server_base.cc:1061] running on GCE node
I20261118 02:54:57.395190  3383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:54:57.396548  3383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:54:57.397734  3383 hybrid_clock.cc:648] HybridClock initialized: now 1794970497397681 us; error 43 us; skew 500 ppm
I20261118 02:54:57.398057  3383 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:54:57.400012  3383 webserver.cc:577] Webserver started at http://127.2.229.124:39557/ using document root <none> and password file <none>
I20261118 02:54:57.400686  3383 fs_manager.cc:362] Metadata directory not provided
I20261118 02:54:57.400769  3383 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:54:57.401073  3383 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 02:54:57.403333  3383 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/instance:
uuid: "813a6937cea442ef81645fccc32207f4"
format_stamp: "Formatted at 2026-11-18 02:54:57 on dist-test-slave-49p1"
I20261118 02:54:57.404026  3383 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal/instance:
uuid: "813a6937cea442ef81645fccc32207f4"
format_stamp: "Formatted at 2026-11-18 02:54:57 on dist-test-slave-49p1"
I20261118 02:54:57.409183  3383 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20261118 02:54:57.411052  3382 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } attempt: 4
I20261118 02:54:57.412738  3400 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
W20261118 02:54:57.413053  3382 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:54:57.414247  3383 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20261118 02:54:57.414533  3383 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
uuid: "813a6937cea442ef81645fccc32207f4"
format_stamp: "Formatted at 2026-11-18 02:54:57 on dist-test-slave-49p1"
I20261118 02:54:57.414742  3383 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:54:57.428887  3383 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:54:57.429796  3383 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:54:57.430063  3383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:54:57.450080  3383 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.124:36695
I20261118 02:54:57.450137  3451 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.124:36695 every 8 connection(s)
I20261118 02:54:57.451520  3383 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
I20261118 02:54:57.453918  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 3383
I20261118 02:54:57.454114  2965 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal/instance
I20261118 02:54:57.456431  3452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:54:57.463096  3452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } has no permanent_uuid. Determining permanent_uuid...
I20261118 02:54:57.475638  3452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } has no permanent_uuid. Determining permanent_uuid...
I20261118 02:54:57.483482  3452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } has no permanent_uuid. Determining permanent_uuid...
I20261118 02:54:57.486673  3312 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } attempt: 3
I20261118 02:54:57.497180  3452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap starting.
I20261118 02:54:57.498615  3312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Bootstrap starting.
I20261118 02:54:57.500439  3452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Neither blocks nor log segments found. Creating new log.
I20261118 02:54:57.501435  3452 log.cc:826] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Log is configured to *not* fsync() on all Append() calls
I20261118 02:54:57.501684  3312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Neither blocks nor log segments found. Creating new log.
I20261118 02:54:57.502955  3312 log.cc:826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Log is configured to *not* fsync() on all Append() calls
I20261118 02:54:57.504689  3452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: No bootstrap required, opened a new log
I20261118 02:54:57.505167  3312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: No bootstrap required, opened a new log
I20261118 02:54:57.510484  3452 raft_consensus.cc:360] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.510754  3452 raft_consensus.cc:386] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:54:57.510862  3452 raft_consensus.cc:748] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813a6937cea442ef81645fccc32207f4, State: Initialized, Role: FOLLOWER
I20261118 02:54:57.510764  3312 raft_consensus.cc:360] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.511034  3312 raft_consensus.cc:386] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:54:57.511140  3312 raft_consensus.cc:748] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Initialized, Role: FOLLOWER
I20261118 02:54:57.511459  3452 consensus_queue.cc:261] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.511744  3312 consensus_queue.cc:261] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.512408  3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.512722  3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:57.513617  3452 sys_catalog.cc:565] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:54:57.513945  3312 sys_catalog.cc:565] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:54:57.512709  3463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.514487  3463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
W20261118 02:54:57.530668  3487 catalog_manager.cc:1609] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20261118 02:54:57.530831  3487 catalog_manager.cc:898] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20261118 02:54:57.526643  3246 master_runner.cc:187] Error getting master registration for 127.2.229.125:38339: OK, instance_id { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" instance_seqno: 1794970497004867 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20261118 02:54:57.534158  3246 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:54:57.535902  3266 tablet.cc:2441] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20261118 02:54:57.536818  3493 catalog_manager.cc:1609] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20261118 02:54:57.536950  3493 catalog_manager.cc:898] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20261118 02:54:57.537336  3383 master_runner.cc:187] Error getting master registration for 127.2.229.125:38339: OK, instance_id { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" instance_seqno: 1794970497004867 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20261118 02:54:57.543740  3383 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:54:57.601938  3463 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:54:57.602249  3463 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.604494  3463 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:54:57.605849  3427 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4" is_pre_election: true
I20261118 02:54:57.606558  3427 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 0.
I20261118 02:54:57.607404  3263 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4; no voters: 
I20261118 02:54:57.608546  3463 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:54:57.608690  3463 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:54:57.608775  3463 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:54:57.612150  3463 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.623270  3427 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4"
I20261118 02:54:57.623477  3427 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:54:57.627024  3427 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 1.
I20261118 02:54:57.627271  3463 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 1 election: Requested vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:54:57.628038  3263 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4; no voters: 
I20261118 02:54:57.628651  3463 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:54:57.652120  3463 raft_consensus.cc:705] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 1 LEADER]: Becoming Leader. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Running, Role: LEADER
I20261118 02:54:57.653203  3463 consensus_queue.cc:238] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.675707  3496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 1 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.676425  3496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
W20261118 02:54:57.682088  3406 tablet.cc:2441] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 02:54:57.684000  3500 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:54:57.686430  3500 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:54:57.686980  3496 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:54:57.706310  3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 1 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.706588  3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:57.712242  3382 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } attempt: 5
I20261118 02:54:57.717248  3496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.717730  3496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:54:57.730195  3382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap starting.
I20261118 02:54:57.770082  3382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Neither blocks nor log segments found. Creating new log.
I20261118 02:54:57.778677  3382 log.cc:826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Log is configured to *not* fsync() on all Append() calls
I20261118 02:54:57.786593  3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.786809  3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:57.807310  3382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: No bootstrap required, opened a new log
I20261118 02:54:57.810382  3500 catalog_manager.cc:1398] Generated new cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:54:57.810503  3500 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:54:57.836112  3500 catalog_manager.cc:1421] Generated new certificate authority record
I20261118 02:54:57.838007  3500 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:54:57.843020  3382 raft_consensus.cc:360] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.844609  3382 raft_consensus.cc:386] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:54:57.845155  3382 raft_consensus.cc:748] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Initialized, Role: FOLLOWER
I20261118 02:54:57.848489  3382 consensus_queue.cc:261] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:57.859978  3500 catalog_manager.cc:6221] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Generated new TSK 0
I20261118 02:54:57.861334  3500 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:54:57.862920  3500 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:54:57.864259  3500 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 02:54:57.868654  3508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.868974  3508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:57.870929  3382 sys_catalog.cc:565] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:54:57.871308  3355 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:54:57.875062  3355 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 1 FOLLOWER]: Refusing update from remote peer 26c6b20fa203404a88463ea27bdd3c7d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20261118 02:54:57.875769  3357 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b" is_pre_election: true
I20261118 02:54:57.876155  3499 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:54:57.876289  3357 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 1.
I20261118 02:54:57.876751  3356 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b"
I20261118 02:54:57.889021  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.229.65:0
--local_ip_for_outbound_sockets=127.2.229.65
--webserver_interface=127.2.229.65
--webserver_port=0
--tserver_master_addrs=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 02:54:57.894387  3356 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 1.
I20261118 02:54:57.952503  3508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 1 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.952473  3522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:57.952821  3508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:57.952841  3522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:58.047472  3529 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:54:58.047583  3529 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: loading cluster ID for follower catalog manager: success
I20261118 02:54:58.051774  3529 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: acquiring CA information for follower catalog manager: success
I20261118 02:54:58.054214  3529 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:54:58.360467  3531 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:54:58.360755  3531 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:58.362532  3531 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
I20261118 02:54:58.362941  3287 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:54:58.363163  3357 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b" is_pre_election: true
I20261118 02:54:58.363919  3403 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:54:58.364320  3531 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:54:58.395880  3522 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:54:58.396143  3522 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:58.397858  3522 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:54:58.398509  3287 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:54:58.398557  3427 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4" is_pre_election: true
I20261118 02:54:58.398777  3427 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 457eb38cba564ecd9f6f11245172260b in term 1.
I20261118 02:54:58.399284  3333 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d
I20261118 02:54:58.399699  3522 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261118 02:54:58.399830  3522 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:54:58.399930  3522 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 1 FOLLOWER]: Advancing to term 2
I20261118 02:54:58.403543  3522 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:58.404337  3522 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 2 election: Requested vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:54:58.404817  3287 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d"
I20261118 02:54:58.404838  3427 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4"
I20261118 02:54:58.405036  3427 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 1 FOLLOWER]: Advancing to term 2
I20261118 02:54:58.407102  3427 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 457eb38cba564ecd9f6f11245172260b in term 2.
I20261118 02:54:58.407513  3333 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d
I20261118 02:54:58.407819  3522 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 2 FOLLOWER]: Leader election won for term 2
I20261118 02:54:58.407998  3522 raft_consensus.cc:705] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 2 LEADER]: Becoming Leader. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Running, Role: LEADER
I20261118 02:54:58.408460  3522 consensus_queue.cc:238] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:58.410373  3508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 2 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:58.410581  3508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:54:58.412915  3536 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:54:58.413828  3427 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 2 FOLLOWER]: Refusing update from remote peer 457eb38cba564ecd9f6f11245172260b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20261118 02:54:58.414433  3514 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:54:58.414793  3514 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:54:58.414903  3514 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20261118 02:54:58.414961  3508 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Connected to new peer: Peer: permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20261118 02:54:58.416915  3531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 2 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:58.417241  3531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:58.418393  3287 raft_consensus.cc:3091] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 1 LEADER]: Stepping down as leader of term 1
I20261118 02:54:58.418519  3287 raft_consensus.cc:748] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Running, Role: LEADER
I20261118 02:54:58.418670  3508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:58.418843  3508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:54:58.418823  3287 consensus_queue.cc:261] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:58.419467  3287 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 1 FOLLOWER]: Advancing to term 2
I20261118 02:54:58.422813  3287 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 2 FOLLOWER]: Refusing update from remote peer 457eb38cba564ecd9f6f11245172260b: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20261118 02:54:58.423769  3508 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20261118 02:54:58.426613  3536 catalog_manager.cc:1535] Initializing Kudu cluster ID...
W20261118 02:54:58.428505  3514 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:54:58.428720  3514 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.229.65
I20261118 02:54:58.430495  3536 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:54:58.431577  3536 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:54:58.433426  3536 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:54:58.433965  3499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 2 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:58.434190  3499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:58.434727  3539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:58.435176  3539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:58.436134  3536 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Loaded TSK: 0
I20261118 02:54:58.437395  3536 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:54:58.438673  3536 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:54:58.439821  3536 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 02:54:58.440634  3531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:58.440830  3531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:58.444355  3514 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.229.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.229.65
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.229.65

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:54:58.446568  3514 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:54:58.449016  3514 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:54:58.460449  3541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:58.462440  3542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:54:58.467446  3514 server_base.cc:1061] running on GCE node
W20261118 02:54:58.467799  3544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:54:58.468505  3514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:54:58.469703  3514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:54:58.470932  3514 hybrid_clock.cc:648] HybridClock initialized: now 1794970498470863 us; error 57 us; skew 500 ppm
I20261118 02:54:58.471282  3514 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:54:58.474207  3514 webserver.cc:577] Webserver started at http://127.2.229.65:33489/ using document root <none> and password file <none>
I20261118 02:54:58.474917  3514 fs_manager.cc:362] Metadata directory not provided
I20261118 02:54:58.475003  3514 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:54:58.475374  3514 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 02:54:58.477813  3514 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/data/instance:
uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
format_stamp: "Formatted at 2026-11-18 02:54:58 on dist-test-slave-49p1"
I20261118 02:54:58.478971  3514 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/instance:
uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
format_stamp: "Formatted at 2026-11-18 02:54:58 on dist-test-slave-49p1"
I20261118 02:54:58.484637  3514 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.005s
I20261118 02:54:58.488225  3550 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:58.489722  3514 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20261118 02:54:58.489877  3514 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal
uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
format_stamp: "Formatted at 2026-11-18 02:54:58 on dist-test-slave-49p1"
I20261118 02:54:58.490063  3514 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:54:58.502089  3514 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:54:58.502875  3514 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:54:58.503103  3514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:54:58.504015  3514 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 02:54:58.505708  3514 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 02:54:58.505801  3514 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:58.505897  3514 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 02:54:58.505963  3514 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:58.534147  3487 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:54:58.539775  3493 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:54:58.539885  3493 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: loading cluster ID for follower catalog manager: success
I20261118 02:54:58.543128  3514 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.65:33705
I20261118 02:54:58.543162  3662 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.65:33705 every 8 connection(s)
I20261118 02:54:58.543691  3493 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: acquiring CA information for follower catalog manager: success
I20261118 02:54:58.544982  3514 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/data/info.pb
I20261118 02:54:58.546111  3493 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:54:58.547616  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 3514
I20261118 02:54:58.547833  2965 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/instance
I20261118 02:54:58.554724  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.229.66:0
--local_ip_for_outbound_sockets=127.2.229.66
--webserver_interface=127.2.229.66
--webserver_port=0
--tserver_master_addrs=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 02:54:58.567888  3665 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:54:58.568353  3665 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:58.569535  3665 heartbeater.cc:508] Master 127.2.229.125:38339 requested a full tablet report, sending...
I20261118 02:54:58.572042  3346 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:54:58.573827  3663 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:54:58.573948  3663 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:58.574318  3663 heartbeater.cc:508] Master 127.2.229.126:38025 requested a full tablet report, sending...
I20261118 02:54:58.574910  3346 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.2.229.65:52789
I20261118 02:54:58.575831  3664 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:54:58.575831  3277 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:54:58.575954  3664 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:58.576434  3664 heartbeater.cc:508] Master 127.2.229.124:36695 requested a full tablet report, sending...
I20261118 02:54:58.578016  3417 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:54:58.895001  3508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:58.895264  3508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
W20261118 02:54:58.903096  3673 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:54:58.903806  3673 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:54:58.903934  3673 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:54:58.912252  3673 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:54:58.912453  3673 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.229.66
I20261118 02:54:58.922938  3673 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.229.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.229.66
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.229.66

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:54:58.924654  3673 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:54:58.926412  3673 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:54:58.936448  3678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:58.936859  3679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:58.937366  3681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:54:58.938285  3673 server_base.cc:1061] running on GCE node
I20261118 02:54:58.938798  3673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:54:58.939705  3673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:54:58.940901  3673 hybrid_clock.cc:648] HybridClock initialized: now 1794970498940848 us; error 49 us; skew 500 ppm
I20261118 02:54:58.941256  3673 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:54:58.943318  3673 webserver.cc:577] Webserver started at http://127.2.229.66:44025/ using document root <none> and password file <none>
I20261118 02:54:58.943980  3673 fs_manager.cc:362] Metadata directory not provided
I20261118 02:54:58.944077  3673 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:54:58.944475  3673 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 02:54:58.946802  3673 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/data/instance:
uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
format_stamp: "Formatted at 2026-11-18 02:54:58 on dist-test-slave-49p1"
I20261118 02:54:58.947548  3673 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/instance:
uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
format_stamp: "Formatted at 2026-11-18 02:54:58 on dist-test-slave-49p1"
I20261118 02:54:58.952814  3673 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20261118 02:54:58.956051  3687 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:58.957516  3673 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20261118 02:54:58.957676  3673 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal
uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
format_stamp: "Formatted at 2026-11-18 02:54:58 on dist-test-slave-49p1"
I20261118 02:54:58.957875  3673 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:54:58.984292  3673 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:54:58.985123  3673 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:54:58.985370  3673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:54:58.986326  3673 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 02:54:58.988174  3673 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 02:54:58.988272  3673 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20261118 02:54:58.988376  3673 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 02:54:58.988443  3673 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:59.041635  3673 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.66:40377
I20261118 02:54:59.041688  3799 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.66:40377 every 8 connection(s)
I20261118 02:54:59.043529  3673 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/data/info.pb
I20261118 02:54:59.044204  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 3673
I20261118 02:54:59.044379  2965 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/instance
I20261118 02:54:59.049989  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.229.67:0
--local_ip_for_outbound_sockets=127.2.229.67
--webserver_interface=127.2.229.67
--webserver_port=0
--tserver_master_addrs=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20261118 02:54:59.064184  3802 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:54:59.064344  3801 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:54:59.064606  3801 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:59.064667  3800 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:54:59.064613  3802 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:59.064791  3800 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:59.065838  3801 heartbeater.cc:508] Master 127.2.229.124:36695 requested a full tablet report, sending...
I20261118 02:54:59.065869  3802 heartbeater.cc:508] Master 127.2.229.125:38339 requested a full tablet report, sending...
I20261118 02:54:59.066675  3800 heartbeater.cc:508] Master 127.2.229.126:38025 requested a full tablet report, sending...
I20261118 02:54:59.067582  3346 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:54:59.067612  3417 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:54:59.067667  3277 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:54:59.068550  3346 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.2.229.66:38401
I20261118 02:54:59.078799  3665 heartbeater.cc:500] Master 127.2.229.125:38339 was elected leader, sending a full tablet report...
I20261118 02:54:59.384109  3808 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:54:59.384342  3808 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:59.385217  3808 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
I20261118 02:54:59.385731  3287 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:54:59.385867  3357 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b" is_pre_election: true
I20261118 02:54:59.386508  3402 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:54:59.386825  3808 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20261118 02:54:59.393323  3807 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:54:59.393580  3807 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:54:59.393699  3807 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20261118 02:54:59.396874  3812 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:54:59.397053  3812 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:59.397723  3812 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:54:59.398237  3357 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b" is_pre_election: true
I20261118 02:54:59.398368  3427 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4" is_pre_election: true
I20261118 02:54:59.398609  3427 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 2.
I20261118 02:54:59.399027  3263 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4; no voters: 457eb38cba564ecd9f6f11245172260b
I20261118 02:54:59.399289  3812 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20261118 02:54:59.399396  3812 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:54:59.399488  3812 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 2 FOLLOWER]: Advancing to term 3
I20261118 02:54:59.401449  3812 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:59.401973  3812 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 3 election: Requested vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:54:59.402426  3357 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b"
I20261118 02:54:59.402446  3427 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4"
I20261118 02:54:59.402653  3427 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 2 FOLLOWER]: Advancing to term 3
W20261118 02:54:59.404016  3807 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:54:59.404276  3807 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.229.67
I20261118 02:54:59.405571  3427 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 3.
I20261118 02:54:59.405942  3263 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4; no voters: 457eb38cba564ecd9f6f11245172260b
I20261118 02:54:59.406261  3812 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 3 FOLLOWER]: Leader election won for term 3
I20261118 02:54:59.406652  3812 raft_consensus.cc:705] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 3 LEADER]: Becoming Leader. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Running, Role: LEADER
I20261118 02:54:59.406944  3812 consensus_queue.cc:238] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:59.407971  3813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:59.408356  3813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:54:59.409422  3427 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 3 FOLLOWER]: Refusing update from remote peer 26c6b20fa203404a88463ea27bdd3c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20261118 02:54:59.415213  3814 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20261118 02:54:59.415520  3357 raft_consensus.cc:3091] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 2 LEADER]: Stepping down as leader of term 2
I20261118 02:54:59.415633  3357 raft_consensus.cc:748] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Running, Role: LEADER
I20261118 02:54:59.415952  3357 consensus_queue.cc:261] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:54:59.416488  3357 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 2 FOLLOWER]: Advancing to term 3
I20261118 02:54:59.419521  3357 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Refusing update from remote peer 26c6b20fa203404a88463ea27bdd3c7d: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20261118 02:54:59.420855  3807 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.229.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.229.67
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.229.67

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:54:59.422152  3808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:59.422389  3808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:59.422694  3807 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:54:59.424437  3814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:59.424744  3814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:54:59.424973  3817 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20261118 02:54:59.425695  3807 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20261118 02:54:59.436048  3816 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:54:59.436496  3808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:59.436740  3808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:59.438357  3824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:59.438558  3816 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:54:59.438495  3820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:59.438706  3820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:59.438733  3824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:54:59.439785  3816 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:54:59.439894  3816 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:54:59.441496  3816 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:54:59.442557  3816 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Loaded TSK: 0
I20261118 02:54:59.443387  3816 catalog_manager.cc:1565] Initializing in-progress tserver states...
W20261118 02:54:59.443599  3825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:54:59.443758  3826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:54:59.444406  3816 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:54:59.445526  3816 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: -1
I20261118 02:54:59.448345  3807 server_base.cc:1061] running on GCE node
W20261118 02:54:59.449287  3828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:54:59.449965  3807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:54:59.451016  3807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:54:59.452230  3807 hybrid_clock.cc:648] HybridClock initialized: now 1794970499452164 us; error 51 us; skew 500 ppm
I20261118 02:54:59.452565  3807 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:54:59.455207  3807 webserver.cc:577] Webserver started at http://127.2.229.67:39695/ using document root <none> and password file <none>
I20261118 02:54:59.455945  3807 fs_manager.cc:362] Metadata directory not provided
I20261118 02:54:59.456046  3807 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:54:59.456454  3807 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20261118 02:54:59.458799  3807 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/data/instance:
uuid: "3f1973986ec1466eab5bec01dd81654f"
format_stamp: "Formatted at 2026-11-18 02:54:59 on dist-test-slave-49p1"
I20261118 02:54:59.459549  3807 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/instance:
uuid: "3f1973986ec1466eab5bec01dd81654f"
format_stamp: "Formatted at 2026-11-18 02:54:59 on dist-test-slave-49p1"
I20261118 02:54:59.465080  3807 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20261118 02:54:59.468488  3834 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:59.469940  3807 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20261118 02:54:59.470105  3807 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal
uuid: "3f1973986ec1466eab5bec01dd81654f"
format_stamp: "Formatted at 2026-11-18 02:54:59 on dist-test-slave-49p1"
I20261118 02:54:59.470289  3807 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:54:59.494714  3807 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:54:59.495630  3807 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:54:59.495878  3807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:54:59.496881  3807 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20261118 02:54:59.498534  3807 ts_tablet_manager.cc:631] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20261118 02:54:59.498627  3807 ts_tablet_manager.cc:577] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:59.498729  3807 ts_tablet_manager.cc:662] Registered 0 tablets
I20261118 02:54:59.498799  3807 ts_tablet_manager.cc:641] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:54:59.535753  3807 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.67:42237
I20261118 02:54:59.535802  3946 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.67:42237 every 8 connection(s)
I20261118 02:54:59.537293  3807 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/data/info.pb
I20261118 02:54:59.540560  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 3807
I20261118 02:54:59.540755  2965 external_mini_cluster.cc:1448] Reading /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/instance
I20261118 02:54:59.555756  3949 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:54:59.556232  3949 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:59.557366  3949 heartbeater.cc:508] Master 127.2.229.125:38339 requested a full tablet report, sending...
I20261118 02:54:59.559212  3346 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:54:59.561666  3948 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:54:59.561831  3948 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:59.562374  3948 heartbeater.cc:508] Master 127.2.229.124:36695 requested a full tablet report, sending...
I20261118 02:54:59.563476  3417 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:54:59.563777  3947 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:54:59.563903  3947 heartbeater.cc:462] Registering TS with master...
I20261118 02:54:59.564309  3947 heartbeater.cc:508] Master 127.2.229.126:38025 requested a full tablet report, sending...
I20261118 02:54:59.565203  3277 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:54:59.567343  3277 master_service.cc:507] Signed X509 certificate for tserver {username='slave'} at 127.2.229.67:38945
I20261118 02:54:59.570621  3802 heartbeater.cc:500] Master 127.2.229.125:38339 was elected leader, sending a full tablet report...
I20261118 02:54:59.575372  2965 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20261118 02:54:59.609403  3277 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_dc3979f8c8b148caa37b0a8ef420ba9c"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.609889  3276 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_df2a47bb353f4cdcb4c16399a99cb722"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.610028  3274 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_0006608f28004098bb49ed40308f98fe"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.610365  3273 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_f777ce1b15a4431fa208932de26f41fb"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.610468  3272 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_295a3e7e0d1f4e9db7078f6e5a5b9da1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.611837  3270 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_bca0059ea2004174b89866f3e8cc68fb"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.612268  3268 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_287099a68a3540f1be0284c71d4d7056"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.613899  3271 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_a4f08da750c54c49b9b7d73ebcf52624"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.615154  3275 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_c113c2b57e17479e92e241234a241af7"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:54:59.614686  3269 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_28c62f411da74545aa42dad7325bc064"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20261118 02:54:59.619489  3277 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_dc3979f8c8b148caa37b0a8ef420ba9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 02:54:59.636780  2965 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task8fD3U4/build/asan/bin/kudu pid 3383
I20261118 02:54:59.638343  3447 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 02:54:59.815558  2965 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task8fD3U4/build/asan/bin/kudu with pid 3383
I20261118 02:54:59.842792  3817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:54:59.843138  3817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
W20261118 02:54:59.940655  3263 consensus_peers.cc:597] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d -> Peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Couldn't send request to peer 813a6937cea442ef81645fccc32207f4. Status: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:00.052235  3277 catalog_manager.cc:2215] created HMS catalog entry for table  [id=052f3134115244a6a2443e96121313e6]
W20261118 02:55:00.057281  3274 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0006608f28004098bb49ed40308f98fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:00.058568  3276 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_df2a47bb353f4cdcb4c16399a99cb722 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:00.059717  3273 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f777ce1b15a4431fa208932de26f41fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:00.068867  3948 heartbeater.cc:648] Failed to heartbeat to 127.2.229.124:36695 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:00.070811  3947 heartbeater.cc:500] Master 127.2.229.126:38025 was elected leader, sending a full tablet report...
I20261118 02:55:00.072726  3800 heartbeater.cc:500] Master 127.2.229.126:38025 was elected leader, sending a full tablet report...
W20261118 02:55:00.075165  3801 heartbeater.cc:648] Failed to heartbeat to 127.2.229.124:36695 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:00.081283  3663 heartbeater.cc:500] Master 127.2.229.126:38025 was elected leader, sending a full tablet report...
W20261118 02:55:00.087898  3664 heartbeater.cc:648] Failed to heartbeat to 127.2.229.124:36695 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:00.123944  3277 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_dc3979f8c8b148caa37b0a8ef420ba9c" } new_table_name: "default.Table_8ae376d4d4d64414b5c08dc3c2e58bc4" modify_external_catalogs: true
W20261118 02:55:00.141810  3272 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_295a3e7e0d1f4e9db7078f6e5a5b9da1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:00.142956  3270 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_bca0059ea2004174b89866f3e8cc68fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:00.143929  3271 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a4f08da750c54c49b9b7d73ebcf52624 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:00.144840  3275 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_c113c2b57e17479e92e241234a241af7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:00.145725  3268 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_287099a68a3540f1be0284c71d4d7056 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:00.146646  3269 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_28c62f411da74545aa42dad7325bc064 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 02:55:00.149622  3274 catalog_manager.cc:2215] created HMS catalog entry for table  [id=a2ece7af36934cdf80ad8458024adec7]
I20261118 02:55:00.217029  3274 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_0006608f28004098bb49ed40308f98fe" } new_table_name: "default.Table_4f7796d5d5d54def843b971e66d15ea6" modify_external_catalogs: true
I20261118 02:55:00.230227  3276 catalog_manager.cc:2215] created HMS catalog entry for table  [id=8a5ea4b95bee4db7af9ba875250d564b]
I20261118 02:55:00.244421  3276 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_df2a47bb353f4cdcb4c16399a99cb722" } new_table_name: "default.Table_0385cef6593c419eb426657c8cbef2bf" modify_external_catalogs: true
I20261118 02:55:00.245314  3881 tablet_service.cc:1500] Processing CreateTablet for tablet 9da387134c724364a6c46f65f4e3d605 (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.246308  3882 tablet_service.cc:1500] Processing CreateTablet for tablet d35ba6ef82f9493ca4605294f109b5dc (DEFAULT_TABLE table=default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.247421  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35ba6ef82f9493ca4605294f109b5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.247543  3598 tablet_service.cc:1500] Processing CreateTablet for tablet d35ba6ef82f9493ca4605294f109b5dc (DEFAULT_TABLE table=default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.249102  3597 tablet_service.cc:1500] Processing CreateTablet for tablet 9da387134c724364a6c46f65f4e3d605 (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.249977  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da387134c724364a6c46f65f4e3d605. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.252460  3880 tablet_service.cc:1500] Processing CreateTablet for tablet 1d347c0ae53241708ec623562f4740d4 (DEFAULT_TABLE table=default.table_df2a47bb353f4cdcb4c16399a99cb722 [id=8a5ea4b95bee4db7af9ba875250d564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.252686  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da387134c724364a6c46f65f4e3d605. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.253087  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d347c0ae53241708ec623562f4740d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.256911  3596 tablet_service.cc:1500] Processing CreateTablet for tablet 1d347c0ae53241708ec623562f4740d4 (DEFAULT_TABLE table=default.table_df2a47bb353f4cdcb4c16399a99cb722 [id=8a5ea4b95bee4db7af9ba875250d564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.257423  3596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d347c0ae53241708ec623562f4740d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.266620  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35ba6ef82f9493ca4605294f109b5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.275245  3735 tablet_service.cc:1500] Processing CreateTablet for tablet d35ba6ef82f9493ca4605294f109b5dc (DEFAULT_TABLE table=default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.276314  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 9da387134c724364a6c46f65f4e3d605 (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.276580  3733 tablet_service.cc:1500] Processing CreateTablet for tablet 1d347c0ae53241708ec623562f4740d4 (DEFAULT_TABLE table=default.table_df2a47bb353f4cdcb4c16399a99cb722 [id=8a5ea4b95bee4db7af9ba875250d564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.277709  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d347c0ae53241708ec623562f4740d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.278225  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9da387134c724364a6c46f65f4e3d605. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.280815  4012 tablet_bootstrap.cc:492] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.281162  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d35ba6ef82f9493ca4605294f109b5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.288311  4012 tablet_bootstrap.cc:654] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.290895  4011 tablet_bootstrap.cc:492] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.294698  4011 tablet_bootstrap.cc:654] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.295320  4012 log.cc:826] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:00.297165  4013 tablet_bootstrap.cc:492] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.297922  4011 log.cc:826] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:00.300823  4013 tablet_bootstrap.cc:654] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.302726  4013 log.cc:826] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:00.314244  4011 tablet_bootstrap.cc:492] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.314612  4011 ts_tablet_manager.cc:1449] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.002s
I20261118 02:55:00.317462  4012 tablet_bootstrap.cc:492] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.320417  4012 ts_tablet_manager.cc:1449] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.005s
I20261118 02:55:00.321329  4013 tablet_bootstrap.cc:492] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.321704  4013 ts_tablet_manager.cc:1449] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.004s
I20261118 02:55:00.327342  4012 raft_consensus.cc:360] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.327785  4012 raft_consensus.cc:386] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.327900  4012 raft_consensus.cc:748] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.329339  4012 consensus_queue.cc:261] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.329422  4013 raft_consensus.cc:360] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.330000  4013 raft_consensus.cc:386] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.330142  4013 raft_consensus.cc:748] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.330996  4013 consensus_queue.cc:261] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.335538  3273 catalog_manager.cc:2215] created HMS catalog entry for table  [id=4a09f210503b499f8e842f6daafb33da]
I20261118 02:55:00.349604  3273 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_22f31dd1ff9e4185a6618f11b5493434"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:00.350908  4011 raft_consensus.cc:360] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.351361  4011 raft_consensus.cc:386] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.351476  4011 raft_consensus.cc:748] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.352247  4011 consensus_queue.cc:261] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.358886  3882 tablet_service.cc:1500] Processing CreateTablet for tablet 4c230b673b8f487fbc79bdfb11a48628 (DEFAULT_TABLE table=default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.358867  3596 tablet_service.cc:1500] Processing CreateTablet for tablet 4c230b673b8f487fbc79bdfb11a48628 (DEFAULT_TABLE table=default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.359292  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c230b673b8f487fbc79bdfb11a48628. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.359292  3596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c230b673b8f487fbc79bdfb11a48628. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.359711  4011 ts_tablet_manager.cc:1480] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.008s
I20261118 02:55:00.360901  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 4c230b673b8f487fbc79bdfb11a48628 (DEFAULT_TABLE table=default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.361301  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c230b673b8f487fbc79bdfb11a48628. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.363598  4011 tablet_bootstrap.cc:492] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.365597  4011 tablet_bootstrap.cc:654] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.369576  4012 ts_tablet_manager.cc:1480] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.049s	user 0.014s	sys 0.001s
I20261118 02:55:00.372473  4012 tablet_bootstrap.cc:492] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.374375  4012 tablet_bootstrap.cc:654] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.376399  4011 tablet_bootstrap.cc:492] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.376606  4011 ts_tablet_manager.cc:1449] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.013s	user 0.007s	sys 0.001s
I20261118 02:55:00.377655  4013 ts_tablet_manager.cc:1480] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.056s	user 0.010s	sys 0.013s
I20261118 02:55:00.378535  3813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:00.378729  3813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:00.378830  4013 tablet_bootstrap.cc:492] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.383507  4013 tablet_bootstrap.cc:654] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.383839  4011 raft_consensus.cc:360] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.384505  4011 raft_consensus.cc:386] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.384788  4011 raft_consensus.cc:748] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.386204  4011 consensus_queue.cc:261] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.388906  4011 ts_tablet_manager.cc:1480] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.000s
I20261118 02:55:00.390585  4011 tablet_bootstrap.cc:492] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.393182  4011 tablet_bootstrap.cc:654] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.393857  4012 tablet_bootstrap.cc:492] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.394022  4012 ts_tablet_manager.cc:1449] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.000s
I20261118 02:55:00.394488  4013 tablet_bootstrap.cc:492] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.394627  4013 ts_tablet_manager.cc:1449] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.016s	user 0.004s	sys 0.004s
I20261118 02:55:00.395565  4013 raft_consensus.cc:360] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.395723  4013 raft_consensus.cc:386] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.395813  4013 raft_consensus.cc:748] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.396037  4013 consensus_queue.cc:261] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.396378  4012 raft_consensus.cc:360] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.396605  4013 ts_tablet_manager.cc:1480] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:00.396615  4012 raft_consensus.cc:386] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.396886  4013 tablet_bootstrap.cc:492] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.397018  4012 raft_consensus.cc:748] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.397346  4012 consensus_queue.cc:261] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.397943  4012 ts_tablet_manager.cc:1480] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 02:55:00.398690  4013 tablet_bootstrap.cc:654] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.400358  4012 tablet_bootstrap.cc:492] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.401256  4011 tablet_bootstrap.cc:492] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.401438  4011 ts_tablet_manager.cc:1449] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.011s	user 0.002s	sys 0.006s
I20261118 02:55:00.402194  4011 raft_consensus.cc:360] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.402381  4011 raft_consensus.cc:386] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.402462  4011 raft_consensus.cc:748] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.402734  4011 consensus_queue.cc:261] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.403929  4012 tablet_bootstrap.cc:654] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.405066  4011 ts_tablet_manager.cc:1480] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.001s
I20261118 02:55:00.407989  4011 tablet_bootstrap.cc:492] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.409744  4011 tablet_bootstrap.cc:654] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.412372  4011 tablet_bootstrap.cc:492] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.412524  4011 ts_tablet_manager.cc:1449] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20261118 02:55:00.413409  4011 raft_consensus.cc:360] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.413542  4011 raft_consensus.cc:386] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.413595  4011 raft_consensus.cc:748] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.413780  4011 consensus_queue.cc:261] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.414228  4011 ts_tablet_manager.cc:1480] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 02:55:00.415045  4013 tablet_bootstrap.cc:492] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.415215  4013 ts_tablet_manager.cc:1449] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.018s	user 0.002s	sys 0.012s
I20261118 02:55:00.417480  4013 raft_consensus.cc:360] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.417682  4013 raft_consensus.cc:386] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.417820  4013 raft_consensus.cc:748] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.418079  4013 consensus_queue.cc:261] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.418542  4013 ts_tablet_manager.cc:1480] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.003s	user 0.001s	sys 0.001s
I20261118 02:55:00.418869  4013 tablet_bootstrap.cc:492] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.420616  4013 tablet_bootstrap.cc:654] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.424837  4012 tablet_bootstrap.cc:492] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.424872  4013 tablet_bootstrap.cc:492] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.424998  4013 ts_tablet_manager.cc:1449] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20261118 02:55:00.425041  4012 ts_tablet_manager.cc:1449] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.025s	user 0.003s	sys 0.013s
I20261118 02:55:00.425782  4012 raft_consensus.cc:360] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.425794  4013 raft_consensus.cc:360] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.425967  4013 raft_consensus.cc:386] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.425995  4012 raft_consensus.cc:386] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.426057  4013 raft_consensus.cc:748] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.426084  4012 raft_consensus.cc:748] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.426292  4013 consensus_queue.cc:261] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.426332  4012 consensus_queue.cc:261] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.426821  4012 ts_tablet_manager.cc:1480] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 02:55:00.427209  4012 tablet_bootstrap.cc:492] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.429081  4012 tablet_bootstrap.cc:654] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.431933  4013 ts_tablet_manager.cc:1480] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.001s
I20261118 02:55:00.437523  4017 raft_consensus.cc:499] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:00.437824  4017 raft_consensus.cc:521] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.438512  4012 tablet_bootstrap.cc:492] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.438681  4012 ts_tablet_manager.cc:1449] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.004s
I20261118 02:55:00.439566  4012 raft_consensus.cc:360] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.439760  4012 raft_consensus.cc:386] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.439862  4012 raft_consensus.cc:748] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.440130  4012 consensus_queue.cc:261] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.441303  4017 leader_election.cc:290] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:00.450096  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1d347c0ae53241708ec623562f4740d4" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:00.450697  3755 raft_consensus.cc:2490] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:00.451416  3552 leader_election.cc:304] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:00.451931  4017 raft_consensus.cc:2826] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:00.452076  4017 raft_consensus.cc:499] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:00.452214  4017 raft_consensus.cc:3096] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.455505  4017 raft_consensus.cc:521] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.455982  3272 catalog_manager.cc:2215] created HMS catalog entry for table  [id=44b390866720455b8d2714eae004383b]
I20261118 02:55:00.456753  4017 leader_election.cc:290] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:00.457056  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1d347c0ae53241708ec623562f4740d4" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:00.457269  3755 raft_consensus.cc:3096] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.460831  3755 raft_consensus.cc:2490] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:00.461428  3552 leader_election.cc:304] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:00.461755  4017 raft_consensus.cc:2826] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:00.461974  4017 raft_consensus.cc:705] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:00.462618  4017 consensus_queue.cc:238] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.467639  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1d347c0ae53241708ec623562f4740d4" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:00.468195  3902 raft_consensus.cc:2490] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:00.469256  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1d347c0ae53241708ec623562f4740d4" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:00.469559  3901 raft_consensus.cc:3096] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.479483  4012 ts_tablet_manager.cc:1480] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.041s	user 0.019s	sys 0.019s
I20261118 02:55:00.482509  3901 raft_consensus.cc:2490] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:00.487835  3734 tablet_service.cc:1500] Processing CreateTablet for tablet a95a3f25901e468ab6e6cd02c2cd035f (DEFAULT_TABLE table=default.table_295a3e7e0d1f4e9db7078f6e5a5b9da1 [id=44b390866720455b8d2714eae004383b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.488382  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95a3f25901e468ab6e6cd02c2cd035f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.492393  4017 consensus_queue.cc:1049] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:00.493752  3272 master_service.cc:965] ReplaceTablet: received request to replace tablet 9da387134c724364a6c46f65f4e3d605 from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:00.494998  3882 tablet_service.cc:1500] Processing CreateTablet for tablet a95a3f25901e468ab6e6cd02c2cd035f (DEFAULT_TABLE table=default.table_295a3e7e0d1f4e9db7078f6e5a5b9da1 [id=44b390866720455b8d2714eae004383b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.495446  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95a3f25901e468ab6e6cd02c2cd035f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.496140  3596 tablet_service.cc:1500] Processing CreateTablet for tablet a95a3f25901e468ab6e6cd02c2cd035f (DEFAULT_TABLE table=default.table_295a3e7e0d1f4e9db7078f6e5a5b9da1 [id=44b390866720455b8d2714eae004383b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.496567  3596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95a3f25901e468ab6e6cd02c2cd035f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.497073  4017 consensus_queue.cc:1049] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:00.502254  4013 tablet_bootstrap.cc:492] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.503224  4012 tablet_bootstrap.cc:492] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.504213  4013 tablet_bootstrap.cc:654] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.505105  4012 tablet_bootstrap.cc:654] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.510735  3272 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet 9da387134c724364a6c46f65f4e3d605
I20261118 02:55:00.511519  3272 catalog_manager.cc:6936] ReplaceTablet: tablet 9da387134c724364a6c46f65f4e3d605 deleted and replaced by tablet 6bf520b76802491da8e3b090ad2152da
I20261118 02:55:00.511839  3596 tablet_service.cc:1547] Processing DeleteTablet for tablet 9da387134c724364a6c46f65f4e3d605 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bf520b76802491da8e3b090ad2152da) from {username='slave'} at 127.0.0.1:52990
I20261118 02:55:00.512848  4040 tablet_replica.cc:333] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:00.513280  4040 raft_consensus.cc:2261] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:00.513070  3272 catalog_manager.cc:5840] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20261118 02:55:00.515702  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet 9da387134c724364a6c46f65f4e3d605 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bf520b76802491da8e3b090ad2152da) from {username='slave'} at 127.0.0.1:50234
I20261118 02:55:00.515880  3733 tablet_service.cc:1547] Processing DeleteTablet for tablet 9da387134c724364a6c46f65f4e3d605 with delete_type TABLET_DATA_DELETED (replaced by tablet 6bf520b76802491da8e3b090ad2152da) from {username='slave'} at 127.0.0.1:57880
I20261118 02:55:00.517897  4011 tablet_bootstrap.cc:492] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.519800  4011 tablet_bootstrap.cc:654] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.532027  3596 tablet_service.cc:1500] Processing CreateTablet for tablet 6bf520b76802491da8e3b090ad2152da (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.532541  3596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf520b76802491da8e3b090ad2152da. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.533190  4011 tablet_bootstrap.cc:492] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.533378  4011 ts_tablet_manager.cc:1449] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.016s	user 0.001s	sys 0.003s
I20261118 02:55:00.534143  4011 raft_consensus.cc:360] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.534345  4011 raft_consensus.cc:386] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.534431  4011 raft_consensus.cc:748] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.534718  4040 raft_consensus.cc:2290] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:00.534682  4011 consensus_queue.cc:261] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.535347  4011 ts_tablet_manager.cc:1480] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.001s
I20261118 02:55:00.537659  4042 tablet_replica.cc:333] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:00.538035  4043 tablet_replica.cc:333] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:00.538187  4042 raft_consensus.cc:2261] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:00.538550  4043 raft_consensus.cc:2261] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:00.538619  4042 raft_consensus.cc:2290] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:00.538967  4043 raft_consensus.cc:2290] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:00.540051  4040 ts_tablet_manager.cc:1962] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:00.541092  4012 tablet_bootstrap.cc:492] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.541311  4012 ts_tablet_manager.cc:1449] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.038s	user 0.000s	sys 0.010s
W20261118 02:55:00.540409  3950 tablet.cc:2441] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 02:55:00.543677  4043 ts_tablet_manager.cc:1962] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:00.542219  4012 raft_consensus.cc:360] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.544047  4012 raft_consensus.cc:386] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.544211  4012 raft_consensus.cc:748] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.545596  3733 tablet_service.cc:1500] Processing CreateTablet for tablet 6bf520b76802491da8e3b090ad2152da (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.546067  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf520b76802491da8e3b090ad2152da. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.546165  4042 ts_tablet_manager.cc:1962] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:00.547101  4012 consensus_queue.cc:261] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.553028  4043 ts_tablet_manager.cc:1975] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 02:55:00.553257  4043 log.cc:1199] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/9da387134c724364a6c46f65f4e3d605
I20261118 02:55:00.554076  4043 ts_tablet_manager.cc:1996] T 9da387134c724364a6c46f65f4e3d605 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:00.554884  3272 master_service.cc:965] ReplaceTablet: received request to replace tablet d35ba6ef82f9493ca4605294f109b5dc from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:00.556187  3882 tablet_service.cc:1500] Processing CreateTablet for tablet 6bf520b76802491da8e3b090ad2152da (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.556603  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf520b76802491da8e3b090ad2152da. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.556599  3262 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet 9da387134c724364a6c46f65f4e3d605 (table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
W20261118 02:55:00.557857  3803 tablet.cc:2441] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 02:55:00.561553  3272 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet d35ba6ef82f9493ca4605294f109b5dc
I20261118 02:55:00.562235  3733 tablet_service.cc:1547] Processing DeleteTablet for tablet d35ba6ef82f9493ca4605294f109b5dc with delete_type TABLET_DATA_DELETED (replaced by tablet 4ea7c6ad69d94b06bb0e72468db5c834) from {username='slave'} at 127.0.0.1:57880
I20261118 02:55:00.562553  4043 tablet_replica.cc:333] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:00.562843  4043 raft_consensus.cc:2261] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:00.562845  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet d35ba6ef82f9493ca4605294f109b5dc with delete_type TABLET_DATA_DELETED (replaced by tablet 4ea7c6ad69d94b06bb0e72468db5c834) from {username='slave'} at 127.0.0.1:50234
I20261118 02:55:00.563090  4043 raft_consensus.cc:2290] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:00.563697  4018 raft_consensus.cc:499] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:00.563844  4043 ts_tablet_manager.cc:1962] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:00.563867  4018 raft_consensus.cc:521] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.565840  4018 leader_election.cc:290] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:00.566346  3272 catalog_manager.cc:6936] ReplaceTablet: tablet d35ba6ef82f9493ca4605294f109b5dc deleted and replaced by tablet 4ea7c6ad69d94b06bb0e72468db5c834
I20261118 02:55:00.568436  3272 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_f777ce1b15a4431fa208932de26f41fb" } new_table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" modify_external_catalogs: true
I20261118 02:55:00.570577  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.124:36695
--webserver_interface=127.2.229.124
--webserver_port=39557
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 02:55:00.570830  4043 ts_tablet_manager.cc:1975] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 02:55:00.570952  4043 log.cc:1199] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/d35ba6ef82f9493ca4605294f109b5dc
I20261118 02:55:00.571225  4043 ts_tablet_manager.cc:1996] T d35ba6ef82f9493ca4605294f109b5dc P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:00.573207  3262 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet d35ba6ef82f9493ca4605294f109b5dc (table default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]) successfully deleted
I20261118 02:55:00.580549  3733 tablet_service.cc:1500] Processing CreateTablet for tablet 4ea7c6ad69d94b06bb0e72468db5c834 (DEFAULT_TABLE table=default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.581049  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ea7c6ad69d94b06bb0e72468db5c834. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.583196  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4c230b673b8f487fbc79bdfb11a48628" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:00.583457  3902 raft_consensus.cc:2490] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:00.584059  3690 leader_election.cc:304] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:00.584497  4018 raft_consensus.cc:2826] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:00.584605  4018 raft_consensus.cc:499] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:00.584692  4018 raft_consensus.cc:3096] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.585318  3881 tablet_service.cc:1500] Processing CreateTablet for tablet 4ea7c6ad69d94b06bb0e72468db5c834 (DEFAULT_TABLE table=default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.585775  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ea7c6ad69d94b06bb0e72468db5c834. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.586014  4013 tablet_bootstrap.cc:492] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.586190  4013 ts_tablet_manager.cc:1449] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.084s	user 0.002s	sys 0.006s
I20261118 02:55:00.586987  4013 raft_consensus.cc:360] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.587194  4013 raft_consensus.cc:386] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.587275  4013 raft_consensus.cc:748] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.587484  4018 raft_consensus.cc:521] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.587584  4013 consensus_queue.cc:261] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.588265  4018 leader_election.cc:290] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:00.589015  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4c230b673b8f487fbc79bdfb11a48628" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:00.589186  3902 raft_consensus.cc:3096] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.592068  3902 raft_consensus.cc:2490] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:00.592625  3690 leader_election.cc:304] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:00.595315  4018 raft_consensus.cc:2826] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:00.595502  4018 raft_consensus.cc:705] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:00.596016  4018 consensus_queue.cc:238] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.596525  4011 tablet_bootstrap.cc:492] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.598263  4011 tablet_bootstrap.cc:654] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
W20261118 02:55:00.602990  3666 tablet.cc:2441] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20261118 02:55:00.603596  4012 ts_tablet_manager.cc:1480] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.062s	user 0.002s	sys 0.000s
I20261118 02:55:00.607456  3598 tablet_service.cc:1547] Processing DeleteTablet for tablet d35ba6ef82f9493ca4605294f109b5dc with delete_type TABLET_DATA_DELETED (replaced by tablet 4ea7c6ad69d94b06bb0e72468db5c834) from {username='slave'} at 127.0.0.1:52990
I20261118 02:55:00.607466  3270 catalog_manager.cc:2215] created HMS catalog entry for table  [id=469b750094124f159d835e6fb5a6486e]
I20261118 02:55:00.608688  3597 tablet_service.cc:1500] Processing CreateTablet for tablet 4ea7c6ad69d94b06bb0e72468db5c834 (DEFAULT_TABLE table=default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.612509  4013 ts_tablet_manager.cc:1480] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.003s
I20261118 02:55:00.612942  4013 tablet_bootstrap.cc:492] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.614796  4013 tablet_bootstrap.cc:654] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.614872  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ea7c6ad69d94b06bb0e72468db5c834. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.620771  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4c230b673b8f487fbc79bdfb11a48628" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:00.621204  3618 raft_consensus.cc:2490] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:00.621644  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4c230b673b8f487fbc79bdfb11a48628" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:00.621901  3617 raft_consensus.cc:3096] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.632297  4034 raft_consensus.cc:499] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:00.632472  4034 raft_consensus.cc:521] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.633639  3270 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 8) took 1022 ms. Trace:
I20261118 02:55:00.633811  3617 raft_consensus.cc:2490] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:00.633805  3270 rpcz_store.cc:276] 1118 02:54:59.611330 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:54:59.611623 (+   293us) service_pool.cc:224] Handling call
1118 02:55:00.143087 (+531464us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:00.143516 (+   429us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:00.607410 (+463894us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:00.633348 (+ 25938us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:00.633464 (+   116us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:00.633590 (+   126us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:00.608348 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:00.608487 (+   139us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:00.608494 (+     7us) write_op.cc:435] Acquired schema lock
1118 02:55:00.608497 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:00.608614 (+   117us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:00.608629 (+    15us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:00.608635 (+     6us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:00.608735 (+   100us) tablet.cc:709] Row locks acquired
1118 02:55:00.608737 (+     2us) write_op.cc:260] PREPARE: finished
1118 02:55:00.608782 (+    45us) write_op.cc:270] Start()
1118 02:55:00.608820 (+    38us) write_op.cc:276] Timestamp: P: 1794970500608749 usec, L: 0
1118 02:55:00.608824 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:00.609035 (+   211us) log.cc:844] Serialized 432 byte log entry
1118 02:55:00.630464 (+ 21429us) op_driver.cc:466] REPLICATION: finished
1118 02:55:00.632560 (+  2096us) write_op.cc:301] APPLY: starting
1118 02:55:00.632579 (+    19us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:00.632621 (+    42us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:00.632648 (+    27us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:00.632762 (+   114us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:00.632795 (+    33us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:00.632802 (+     7us) write_op.cc:312] APPLY: finished
1118 02:55:00.632896 (+    94us) log.cc:844] Serialized 28 byte log entry
1118 02:55:00.632958 (+    62us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:00.632978 (+    20us) write_op.cc:454] Released schema lock
1118 02:55:00.632985 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:00.633053 (+    68us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:00.633057 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":312250,"Hive Metastore.run_cpu_time_us":268,"Hive Metastore.run_wall_time_us":151424,"child_traces":[["op",{"apply.queue_time_us":2030,"apply.run_cpu_time_us":492,"apply.run_wall_time_us":491,"num_ops":2,"prepare.queue_time_us":74,"prepare.run_cpu_time_us":817,"prepare.run_wall_time_us":816,"raft.queue_time_us":168,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":307,"replication_time_us":21599}]]}
I20261118 02:55:00.636461  4034 leader_election.cc:290] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:00.641036  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a95a3f25901e468ab6e6cd02c2cd035f" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:00.641340  3902 raft_consensus.cc:2490] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:00.642102  3553 leader_election.cc:304] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:00.642439  4034 raft_consensus.cc:2826] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:00.642531  4034 raft_consensus.cc:499] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:00.642597  4034 raft_consensus.cc:3096] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.644318  3598 tablet_service.cc:1500] Processing CreateTablet for tablet 9a19cb906c974009aa9df99a14a78024 (DEFAULT_TABLE table=default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.644881  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a19cb906c974009aa9df99a14a78024. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.644904  3882 tablet_service.cc:1500] Processing CreateTablet for tablet 9a19cb906c974009aa9df99a14a78024 (DEFAULT_TABLE table=default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.645284  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a19cb906c974009aa9df99a14a78024. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.645689  4034 raft_consensus.cc:521] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.647038  3270 master_service.cc:965] ReplaceTablet: received request to replace tablet d35ba6ef82f9493ca4605294f109b5dc from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:00.647490  4034 leader_election.cc:290] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:00.647536  3270 master_service.cc:965] ReplaceTablet: received request to replace tablet d35ba6ef82f9493ca4605294f109b5dc from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:00.647471  3733 tablet_service.cc:1500] Processing CreateTablet for tablet 9a19cb906c974009aa9df99a14a78024 (DEFAULT_TABLE table=default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.647881  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a19cb906c974009aa9df99a14a78024. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.647936  3270 master_service.cc:965] ReplaceTablet: received request to replace tablet d35ba6ef82f9493ca4605294f109b5dc from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:00.648491  4012 tablet_bootstrap.cc:492] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.648464  3270 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_a14cb6414555446286bf35bc5c6fb910"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:00.650183  4012 tablet_bootstrap.cc:654] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.658237  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a95a3f25901e468ab6e6cd02c2cd035f" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:00.660755  3902 raft_consensus.cc:3096] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.664189  3902 raft_consensus.cc:2490] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:00.664741  3553 leader_election.cc:304] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:00.665130  4034 raft_consensus.cc:2826] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:00.666065  4034 raft_consensus.cc:705] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:00.666419  4034 consensus_queue.cc:238] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.673485  4034 consensus_queue.cc:1049] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:00.700728  4011 tablet_bootstrap.cc:492] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.701049  4011 ts_tablet_manager.cc:1449] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.105s	user 0.021s	sys 0.016s
I20261118 02:55:00.701659  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a95a3f25901e468ab6e6cd02c2cd035f" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:00.701901  3755 raft_consensus.cc:2490] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:00.702028  4011 raft_consensus.cc:360] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.702275  4011 raft_consensus.cc:386] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.702410  4011 raft_consensus.cc:748] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.702527  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a95a3f25901e468ab6e6cd02c2cd035f" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:00.702699  3755 raft_consensus.cc:3096] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.702746  4011 consensus_queue.cc:261] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.704427  4011 ts_tablet_manager.cc:1480] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:00.705394  3755 raft_consensus.cc:2490] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:00.705619  4011 tablet_bootstrap.cc:492] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.707401  4011 tablet_bootstrap.cc:654] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.707796  4034 consensus_queue.cc:1049] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:00.712424  4048 consensus_queue.cc:1049] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:00.717813  4011 tablet_bootstrap.cc:492] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.718004  4011 ts_tablet_manager.cc:1449] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.012s	user 0.006s	sys 0.000s
I20261118 02:55:00.720218  4011 raft_consensus.cc:360] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.720422  4011 raft_consensus.cc:386] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.720497  4011 raft_consensus.cc:748] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.722069  4011 consensus_queue.cc:261] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.722705  3271 catalog_manager.cc:2215] created HMS catalog entry for table  [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:00.723526  4012 tablet_bootstrap.cc:492] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.723672  4012 ts_tablet_manager.cc:1449] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.000s
I20261118 02:55:00.724031  4011 ts_tablet_manager.cc:1480] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 02:55:00.724689  4012 raft_consensus.cc:360] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.724864  4012 raft_consensus.cc:386] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.724951  4012 raft_consensus.cc:748] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.725188  4012 consensus_queue.cc:261] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.725708  4011 tablet_bootstrap.cc:492] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.727444  4011 tablet_bootstrap.cc:654] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.730191  3271 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 7) took 1119 ms. Trace:
I20261118 02:55:00.730332  3271 rpcz_store.cc:276] 1118 02:54:59.611194 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:54:59.613665 (+  2471us) service_pool.cc:224] Handling call
1118 02:55:00.144041 (+530376us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:00.144472 (+   431us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:00.722655 (+578183us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:00.729824 (+  7169us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:00.729998 (+   174us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:00.730141 (+   143us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:00.724490 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:00.724623 (+   133us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:00.724631 (+     8us) write_op.cc:435] Acquired schema lock
1118 02:55:00.724634 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:00.724756 (+   122us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:00.724771 (+    15us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:00.724776 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:00.724858 (+    82us) tablet.cc:709] Row locks acquired
1118 02:55:00.724861 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:00.724904 (+    43us) write_op.cc:270] Start()
1118 02:55:00.724955 (+    51us) write_op.cc:276] Timestamp: P: 1794970500724850 usec, L: 0
1118 02:55:00.724958 (+     3us) op_driver.cc:350] REPLICATION: starting
1118 02:55:00.725147 (+   189us) log.cc:844] Serialized 432 byte log entry
1118 02:55:00.729130 (+  3983us) op_driver.cc:466] REPLICATION: finished
1118 02:55:00.729278 (+   148us) write_op.cc:301] APPLY: starting
1118 02:55:00.729296 (+    18us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:00.729344 (+    48us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:00.729373 (+    29us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:00.729521 (+   148us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:00.729554 (+    33us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:00.729560 (+     6us) write_op.cc:312] APPLY: finished
1118 02:55:00.729633 (+    73us) log.cc:844] Serialized 28 byte log entry
1118 02:55:00.729695 (+    62us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:00.729715 (+    20us) write_op.cc:454] Released schema lock
1118 02:55:00.729722 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:00.729834 (+   112us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:00.729838 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":462805,"Hive Metastore.run_cpu_time_us":249,"Hive Metastore.run_wall_time_us":114981,"child_traces":[["op",{"apply.queue_time_us":99,"apply.run_cpu_time_us":526,"apply.run_wall_time_us":525,"num_ops":2,"prepare.queue_time_us":1036,"prepare.run_cpu_time_us":770,"prepare.run_wall_time_us":772,"raft.queue_time_us":983,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":235,"replication_time_us":4139}]]}
I20261118 02:55:00.731617  3271 catalog_manager.cc:5840] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20261118 02:55:00.733893  4012 ts_tablet_manager.cc:1480] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.000s
I20261118 02:55:00.737948  4011 tablet_bootstrap.cc:492] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.738121  4011 ts_tablet_manager.cc:1449] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.013s	user 0.005s	sys 0.000s
I20261118 02:55:00.738839  4060 consensus_queue.cc:1049] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:00.740178  4012 tablet_bootstrap.cc:492] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.740470  4011 raft_consensus.cc:360] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.741575  4011 raft_consensus.cc:386] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.741676  4011 raft_consensus.cc:748] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.742358  4011 consensus_queue.cc:261] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.744520  4011 ts_tablet_manager.cc:1480] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20261118 02:55:00.748665  3882 tablet_service.cc:1500] Processing CreateTablet for tablet 55f81e7b62e54ec18c7c13a200edb68f (DEFAULT_TABLE table=default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.748814  3733 tablet_service.cc:1500] Processing CreateTablet for tablet 55f81e7b62e54ec18c7c13a200edb68f (DEFAULT_TABLE table=default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.749074  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55f81e7b62e54ec18c7c13a200edb68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.748943  3271 catalog_manager.cc:5840] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66). New cstate: current_term: 1 leader_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20261118 02:55:00.749217  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55f81e7b62e54ec18c7c13a200edb68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.753182  3598 tablet_service.cc:1500] Processing CreateTablet for tablet 55f81e7b62e54ec18c7c13a200edb68f (DEFAULT_TABLE table=default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.753626  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55f81e7b62e54ec18c7c13a200edb68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.754141  4040 ts_tablet_manager.cc:1975] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 02:55:00.754325  4040 log.cc:1199] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/9da387134c724364a6c46f65f4e3d605
I20261118 02:55:00.754830  4040 ts_tablet_manager.cc:1996] T 9da387134c724364a6c46f65f4e3d605 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:00.756652  4013 tablet_bootstrap.cc:492] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.756796  3264 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet 9da387134c724364a6c46f65f4e3d605 (table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
I20261118 02:55:00.756966  4013 ts_tablet_manager.cc:1449] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.144s	user 0.013s	sys 0.029s
I20261118 02:55:00.757735  4013 raft_consensus.cc:360] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.757999  4013 raft_consensus.cc:386] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.758111  4013 raft_consensus.cc:748] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.758447  4013 consensus_queue.cc:261] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.759572  4013 ts_tablet_manager.cc:1480] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20261118 02:55:00.761554  4012 tablet_bootstrap.cc:654] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.761909  4013 tablet_bootstrap.cc:492] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.762741  4011 tablet_bootstrap.cc:492] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.763963  4040 tablet_replica.cc:333] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:00.765817  4011 tablet_bootstrap.cc:654] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.767591  3271 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_87d592a09c3348028719a603956fbce6"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:00.768218  4013 tablet_bootstrap.cc:654] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.769277  4040 raft_consensus.cc:2261] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:00.767479  4042 ts_tablet_manager.cc:1975] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 02:55:00.769922  4042 log.cc:1199] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/9da387134c724364a6c46f65f4e3d605
I20261118 02:55:00.770390  4042 ts_tablet_manager.cc:1996] T 9da387134c724364a6c46f65f4e3d605 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:00.770627  4040 raft_consensus.cc:2290] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:00.772243  3263 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet 9da387134c724364a6c46f65f4e3d605 (table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
I20261118 02:55:00.777318  4040 ts_tablet_manager.cc:1962] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:00.780058  4042 tablet_replica.cc:333] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:00.780643  4042 raft_consensus.cc:2261] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:00.781378  4042 raft_consensus.cc:2290] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:00.787606  4042 ts_tablet_manager.cc:1962] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:00.792671  4012 tablet_bootstrap.cc:492] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.792896  4012 ts_tablet_manager.cc:1449] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.053s	user 0.003s	sys 0.000s
I20261118 02:55:00.793574  4013 tablet_bootstrap.cc:492] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.793830  4013 ts_tablet_manager.cc:1449] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.032s	user 0.003s	sys 0.005s
I20261118 02:55:00.794361  4012 raft_consensus.cc:360] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.794615  4012 raft_consensus.cc:386] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.794698  4012 raft_consensus.cc:748] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.794862  4013 raft_consensus.cc:360] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.795050  4012 consensus_queue.cc:261] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.795126  4013 raft_consensus.cc:386] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.795301  4013 raft_consensus.cc:748] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.795631  4013 consensus_queue.cc:261] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.796227  4013 ts_tablet_manager.cc:1480] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.001s
I20261118 02:55:00.796656  4013 tablet_bootstrap.cc:492] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.800930  4012 ts_tablet_manager.cc:1480] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.000s
I20261118 02:55:00.802672  4013 tablet_bootstrap.cc:654] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.803344  4012 tablet_bootstrap.cc:492] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.805401  4042 ts_tablet_manager.cc:1975] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 02:55:00.806660  4042 log.cc:1199] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/d35ba6ef82f9493ca4605294f109b5dc
I20261118 02:55:00.807137  4042 ts_tablet_manager.cc:1996] T d35ba6ef82f9493ca4605294f109b5dc P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:00.809769  4012 tablet_bootstrap.cc:654] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.809237  4011 tablet_bootstrap.cc:492] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.810150  4011 ts_tablet_manager.cc:1449] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.012s
I20261118 02:55:00.811256  3263 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet d35ba6ef82f9493ca4605294f109b5dc (table default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]) successfully deleted
I20261118 02:55:00.811589  4011 raft_consensus.cc:360] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.811866  4011 raft_consensus.cc:386] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.811971  4011 raft_consensus.cc:748] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.812325  4011 consensus_queue.cc:261] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.816203  4011 ts_tablet_manager.cc:1480] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20261118 02:55:00.824523  4040 ts_tablet_manager.cc:1975] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20261118 02:55:00.824781  4040 log.cc:1199] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/d35ba6ef82f9493ca4605294f109b5dc
I20261118 02:55:00.825371  4040 ts_tablet_manager.cc:1996] T d35ba6ef82f9493ca4605294f109b5dc P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:00.825892  4012 tablet_bootstrap.cc:492] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.826097  4012 ts_tablet_manager.cc:1449] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.023s	user 0.004s	sys 0.000s
I20261118 02:55:00.826701  3264 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet d35ba6ef82f9493ca4605294f109b5dc (table default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6]) successfully deleted
I20261118 02:55:00.826996  4012 raft_consensus.cc:360] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.827170  4012 raft_consensus.cc:386] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.827260  4012 raft_consensus.cc:748] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.827485  4012 consensus_queue.cc:261] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.829828  4012 ts_tablet_manager.cc:1480] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 02:55:00.830137  4012 tablet_bootstrap.cc:492] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.831933  4012 tablet_bootstrap.cc:654] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.836763  4013 tablet_bootstrap.cc:492] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.837028  4013 ts_tablet_manager.cc:1449] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.040s	user 0.005s	sys 0.010s
I20261118 02:55:00.837990  4013 raft_consensus.cc:360] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.838212  4013 raft_consensus.cc:386] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.838152  4012 tablet_bootstrap.cc:492] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.838316  4013 raft_consensus.cc:748] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.838516  4012 ts_tablet_manager.cc:1449] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.008s	user 0.004s	sys 0.000s
I20261118 02:55:00.838630  4013 consensus_queue.cc:261] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.840740  4012 raft_consensus.cc:360] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.841025  4012 raft_consensus.cc:386] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.841150  4012 raft_consensus.cc:748] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.841655  4012 consensus_queue.cc:261] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.841759  4013 ts_tablet_manager.cc:1480] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 02:55:00.844245  4013 tablet_bootstrap.cc:492] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.845513  4012 ts_tablet_manager.cc:1480] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.000s
I20261118 02:55:00.846978  4013 tablet_bootstrap.cc:654] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.852008  4013 tablet_bootstrap.cc:492] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.852231  4013 ts_tablet_manager.cc:1449] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.008s	user 0.006s	sys 0.000s
I20261118 02:55:00.853000  4013 raft_consensus.cc:360] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.853535  3275 catalog_manager.cc:2215] created HMS catalog entry for table  [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:00.853636  4013 raft_consensus.cc:386] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.853713  4013 raft_consensus.cc:748] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.853961  4013 consensus_queue.cc:261] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.854451  4013 ts_tablet_manager.cc:1480] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20261118 02:55:00.864554  3275 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 3) took 1255 ms. Trace:
I20261118 02:55:00.864714  3275 rpcz_store.cc:276] 1118 02:54:59.609056 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:54:59.609636 (+   580us) service_pool.cc:224] Handling call
1118 02:54:59.615092 (+  5456us) spinlock_profiling.cc:245] Waited 5.12 ms on lock 0x616000043354. stack: 00007f9c4434a3a6 00007f9c4434a160 00007f9c4333b34d 00007f9c5a90e704 00007f9c5a90e6cd 00007f9c5386082d 00007f9c53860923 00007f9c503ca221 00007f9c582bf50d 00007f9c5844e873 00007f9c4ed57526 00007f9c4ed57255 00007f9c4b4928de 00007f9c4b4917be 00007f9c4b4979f8 00007f9c4b49835f
1118 02:55:00.144955 (+529863us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:00.145346 (+   391us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:00.853469 (+708123us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:00.864251 (+ 10782us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:00.864379 (+   128us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:00.864500 (+   121us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:00.855867 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:00.856008 (+   141us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:00.856014 (+     6us) write_op.cc:435] Acquired schema lock
1118 02:55:00.856017 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:00.856141 (+   124us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:00.856157 (+    16us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:00.856162 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:00.856248 (+    86us) tablet.cc:709] Row locks acquired
1118 02:55:00.856250 (+     2us) write_op.cc:260] PREPARE: finished
1118 02:55:00.856293 (+    43us) write_op.cc:270] Start()
1118 02:55:00.856329 (+    36us) write_op.cc:276] Timestamp: P: 1794970500856213 usec, L: 0
1118 02:55:00.856333 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:00.856542 (+   209us) log.cc:844] Serialized 432 byte log entry
1118 02:55:00.862786 (+  6244us) op_driver.cc:466] REPLICATION: finished
1118 02:55:00.862941 (+   155us) write_op.cc:301] APPLY: starting
1118 02:55:00.862959 (+    18us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:00.863005 (+    46us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:00.863055 (+    50us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:00.863200 (+   145us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:00.863234 (+    34us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:00.863242 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:00.863352 (+   110us) log.cc:844] Serialized 28 byte log entry
1118 02:55:00.863417 (+    65us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:00.863439 (+    22us) write_op.cc:454] Released schema lock
1118 02:55:00.863445 (+     6us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:00.863515 (+    70us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:00.863519 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":576948,"Hive Metastore.run_cpu_time_us":302,"Hive Metastore.run_wall_time_us":130946,"spinlock_wait_cycles":11774208,"child_traces":[["op",{"apply.queue_time_us":104,"apply.run_cpu_time_us":568,"apply.run_wall_time_us":566,"num_ops":2,"prepare.queue_time_us":1650,"prepare.run_cpu_time_us":815,"prepare.run_wall_time_us":815,"raft.queue_time_us":1881,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":262,"replication_time_us":6416}]]}
I20261118 02:55:00.879803  3882 tablet_service.cc:1500] Processing CreateTablet for tablet 8f853e6b923444efa79cfbab2e31540e (DEFAULT_TABLE table=default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.880312  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f853e6b923444efa79cfbab2e31540e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.880692  3598 tablet_service.cc:1500] Processing CreateTablet for tablet 8f853e6b923444efa79cfbab2e31540e (DEFAULT_TABLE table=default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.881098  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f853e6b923444efa79cfbab2e31540e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.887063  4012 tablet_bootstrap.cc:492] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.887036  4030 raft_consensus.cc:499] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:00.887053  3733 tablet_service.cc:1500] Processing CreateTablet for tablet 8f853e6b923444efa79cfbab2e31540e (DEFAULT_TABLE table=default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.887221  4030 raft_consensus.cc:521] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.887439  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f853e6b923444efa79cfbab2e31540e. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.888222  4030 leader_election.cc:290] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:00.888231  3275 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_295a3e7e0d1f4e9db7078f6e5a5b9da1" } new_table_name: "default.Table_30f082fbe850451b80d98df75ed1081a" modify_external_catalogs: true
I20261118 02:55:00.888932  4012 tablet_bootstrap.cc:654] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.889937  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4ea7c6ad69d94b06bb0e72468db5c834" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:00.890129  3902 raft_consensus.cc:2490] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:00.890599  3553 leader_election.cc:304] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:00.890880  4030 raft_consensus.cc:2826] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:00.890977  4030 raft_consensus.cc:499] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:00.891053  4030 raft_consensus.cc:3096] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.892345  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4ea7c6ad69d94b06bb0e72468db5c834" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:00.892544  3755 raft_consensus.cc:2490] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:00.893671  4030 raft_consensus.cc:521] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.894742  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4ea7c6ad69d94b06bb0e72468db5c834" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:00.894760  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "4ea7c6ad69d94b06bb0e72468db5c834" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:00.894923  3755 raft_consensus.cc:3096] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.894924  3902 raft_consensus.cc:3096] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:00.897768  3902 raft_consensus.cc:2490] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:00.897996  3268 catalog_manager.cc:2215] created HMS catalog entry for table  [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
I20261118 02:55:00.900777  3553 leader_election.cc:304] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:00.901073  4017 raft_consensus.cc:2826] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:00.901223  4017 raft_consensus.cc:705] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:00.901485  4017 consensus_queue.cc:238] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:00.904270  3268 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 10) took 1292 ms. Trace:
I20261118 02:55:00.904412  3268 rpcz_store.cc:276] 1118 02:54:59.611561 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:54:59.612034 (+   473us) service_pool.cc:224] Handling call
1118 02:55:00.145818 (+533784us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:00.146246 (+   428us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:00.897951 (+751705us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:00.903975 (+  6024us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:00.904113 (+   138us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:00.904231 (+   118us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:00.898727 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:00.898853 (+   126us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:00.898859 (+     6us) write_op.cc:435] Acquired schema lock
1118 02:55:00.898861 (+     2us) tablet.cc:670] Decoding operations
1118 02:55:00.898953 (+    92us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:00.898966 (+    13us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:00.898971 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:00.899050 (+    79us) tablet.cc:709] Row locks acquired
1118 02:55:00.899053 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:00.899091 (+    38us) write_op.cc:270] Start()
1118 02:55:00.899122 (+    31us) write_op.cc:276] Timestamp: P: 1794970500899004 usec, L: 0
1118 02:55:00.899125 (+     3us) op_driver.cc:350] REPLICATION: starting
1118 02:55:00.899307 (+   182us) log.cc:844] Serialized 432 byte log entry
1118 02:55:00.902517 (+  3210us) op_driver.cc:466] REPLICATION: finished
1118 02:55:00.902804 (+   287us) write_op.cc:301] APPLY: starting
1118 02:55:00.902822 (+    18us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:00.902879 (+    57us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:00.902913 (+    34us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:00.903031 (+   118us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:00.903064 (+    33us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:00.903071 (+     7us) write_op.cc:312] APPLY: finished
1118 02:55:00.903146 (+    75us) log.cc:844] Serialized 28 byte log entry
1118 02:55:00.903203 (+    57us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:00.903224 (+    21us) write_op.cc:454] Released schema lock
1118 02:55:00.903230 (+     6us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:00.903294 (+    64us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:00.903297 (+     3us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":707074,"Hive Metastore.run_cpu_time_us":269,"Hive Metastore.run_wall_time_us":42372,"child_traces":[["op",{"apply.queue_time_us":231,"apply.run_cpu_time_us":487,"apply.run_wall_time_us":488,"num_ops":2,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":697,"prepare.run_wall_time_us":698,"raft.queue_time_us":356,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":3360}]]}
I20261118 02:55:00.905124  3268 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_2e3917ff98f2474984580e61caa9c17b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:00.909484  3755 raft_consensus.cc:2490] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:00.909751  4030 leader_election.cc:290] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:00.914669  4013 tablet_bootstrap.cc:492] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.916502  4013 tablet_bootstrap.cc:654] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.918707  3881 tablet_service.cc:1500] Processing CreateTablet for tablet ffe8d4c111d14054b76f9c7724072a94 (DEFAULT_TABLE table=default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.919165  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe8d4c111d14054b76f9c7724072a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.921376  4011 tablet_bootstrap.cc:492] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.923242  4011 tablet_bootstrap.cc:654] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.924502  3734 tablet_service.cc:1500] Processing CreateTablet for tablet ffe8d4c111d14054b76f9c7724072a94 (DEFAULT_TABLE table=default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.924963  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe8d4c111d14054b76f9c7724072a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.928500  3598 tablet_service.cc:1500] Processing CreateTablet for tablet ffe8d4c111d14054b76f9c7724072a94 (DEFAULT_TABLE table=default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:00.928941  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffe8d4c111d14054b76f9c7724072a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:00.939474  4030 consensus_queue.cc:1049] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:00.941191  4013 tablet_bootstrap.cc:492] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.941363  4013 ts_tablet_manager.cc:1449] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.027s	user 0.004s	sys 0.007s
I20261118 02:55:00.942764  4013 raft_consensus.cc:360] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.942956  4013 raft_consensus.cc:386] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.943046  4013 raft_consensus.cc:748] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.943265  4013 consensus_queue.cc:261] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.944295  4011 tablet_bootstrap.cc:492] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:00.944451  4011 ts_tablet_manager.cc:1449] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.000s
I20261118 02:55:00.945395  4011 raft_consensus.cc:360] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.945576  4011 raft_consensus.cc:386] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.945667  4011 raft_consensus.cc:748] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.945909  4011 consensus_queue.cc:261] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.947181  4011 ts_tablet_manager.cc:1480] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:00.947479  4011 tablet_bootstrap.cc:492] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:00.949267  4033 consensus_queue.cc:1049] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:00.949764  4012 tablet_bootstrap.cc:492] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:00.949949  4012 ts_tablet_manager.cc:1449] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.063s	user 0.002s	sys 0.018s
I20261118 02:55:00.950810  4012 raft_consensus.cc:360] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.951066  4012 raft_consensus.cc:386] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:00.951202  4012 raft_consensus.cc:748] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:00.951659  4012 consensus_queue.cc:261] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.953037  4012 ts_tablet_manager.cc:1480] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.003s	user 0.001s	sys 0.001s
I20261118 02:55:00.953457  4013 ts_tablet_manager.cc:1480] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.001s
I20261118 02:55:00.953749  4013 tablet_bootstrap.cc:492] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:00.955120  4012 tablet_bootstrap.cc:492] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:00.956234  4011 tablet_bootstrap.cc:654] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.958338  4013 tablet_bootstrap.cc:654] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.959293  4012 tablet_bootstrap.cc:654] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:00.976816  4013 tablet_bootstrap.cc:492] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:00.977128  4019 raft_consensus.cc:499] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:00.977315  4019 raft_consensus.cc:521] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:00.979369  4019 leader_election.cc:290] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:00.997922  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "55f81e7b62e54ec18c7c13a200edb68f" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:00.998242  3617 raft_consensus.cc:2490] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:00.999222  3838 leader_election.cc:304] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:01.000617  4019 raft_consensus.cc:2826] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:01.000756  4019 raft_consensus.cc:499] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:01.000834  4019 raft_consensus.cc:3096] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.009092  4019 raft_consensus.cc:521] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.010766  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "55f81e7b62e54ec18c7c13a200edb68f" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:01.011016  3617 raft_consensus.cc:3096] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.014350  3617 raft_consensus.cc:2490] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:01.015102  3838 leader_election.cc:304] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:01.015421  3269 catalog_manager.cc:2215] created HMS catalog entry for table  [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:01.022222  4019 leader_election.cc:290] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:01.022687  3269 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 9) took 1411 ms. Trace:
I20261118 02:55:01.022745  4019 raft_consensus.cc:2826] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:01.022899  4019 raft_consensus.cc:705] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:01.022835  3269 rpcz_store.cc:276] 1118 02:54:59.611425 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:54:59.614026 (+  2601us) service_pool.cc:224] Handling call
1118 02:55:00.146766 (+532740us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:00.147148 (+   382us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:01.015370 (+868222us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:01.022294 (+  6924us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:01.022512 (+   218us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:01.022640 (+   128us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:01.016315 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:01.016469 (+   154us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:01.016476 (+     7us) write_op.cc:435] Acquired schema lock
1118 02:55:01.016479 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:01.016584 (+   105us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:01.016599 (+    15us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:01.016605 (+     6us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:01.016695 (+    90us) tablet.cc:709] Row locks acquired
1118 02:55:01.016697 (+     2us) write_op.cc:260] PREPARE: finished
1118 02:55:01.016738 (+    41us) write_op.cc:270] Start()
1118 02:55:01.016776 (+    38us) write_op.cc:276] Timestamp: P: 1794970501016627 usec, L: 0
1118 02:55:01.016780 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:01.016985 (+   205us) log.cc:844] Serialized 432 byte log entry
1118 02:55:01.020815 (+  3830us) op_driver.cc:466] REPLICATION: finished
1118 02:55:01.020951 (+   136us) write_op.cc:301] APPLY: starting
1118 02:55:01.020970 (+    19us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:01.021019 (+    49us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:01.021060 (+    41us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:01.021188 (+   128us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:01.021224 (+    36us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:01.021232 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:01.021315 (+    83us) log.cc:844] Serialized 28 byte log entry
1118 02:55:01.021380 (+    65us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:01.021403 (+    23us) write_op.cc:454] Released schema lock
1118 02:55:01.021410 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:01.021479 (+    69us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:01.021483 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":748571,"Hive Metastore.run_cpu_time_us":292,"Hive Metastore.run_wall_time_us":118994,"child_traces":[["op",{"apply.queue_time_us":84,"apply.run_cpu_time_us":519,"apply.run_wall_time_us":519,"num_ops":2,"prepare.queue_time_us":57,"prepare.run_cpu_time_us":792,"prepare.run_wall_time_us":793,"raft.queue_time_us":453,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":308,"replication_time_us":3997}]]}
I20261118 02:55:01.023607  4019 consensus_queue.cc:238] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.034076  3598 tablet_service.cc:1500] Processing CreateTablet for tablet 5f05a8e5b21945a5bd3df86a5467a615 (DEFAULT_TABLE table=default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:01.034158  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 5f05a8e5b21945a5bd3df86a5467a615 (DEFAULT_TABLE table=default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:01.034607  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f05a8e5b21945a5bd3df86a5467a615. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:01.034626  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f05a8e5b21945a5bd3df86a5467a615. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:01.040361  4013 ts_tablet_manager.cc:1449] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.087s	user 0.003s	sys 0.003s
I20261118 02:55:01.041601  3269 catalog_manager.cc:5840] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20261118 02:55:01.042656  4011 tablet_bootstrap.cc:492] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:01.042830  4011 ts_tablet_manager.cc:1449] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.095s	user 0.007s	sys 0.000s
I20261118 02:55:01.043771  4011 raft_consensus.cc:360] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.043972  4011 raft_consensus.cc:386] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:01.044456  4011 raft_consensus.cc:748] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:01.044795  4011 consensus_queue.cc:261] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.048317  4013 raft_consensus.cc:360] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.048539  4013 raft_consensus.cc:386] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:01.048625  4013 raft_consensus.cc:748] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:01.048975  4013 consensus_queue.cc:261] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.050591  4013 ts_tablet_manager.cc:1480] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.000s
I20261118 02:55:01.051680  4013 tablet_bootstrap.cc:492] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:01.053499  4013 tablet_bootstrap.cc:654] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:01.066936  4017 raft_consensus.cc:499] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:01.068428  4017 raft_consensus.cc:521] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.070628  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9a19cb906c974009aa9df99a14a78024" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:01.070953  3755 raft_consensus.cc:2490] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:01.071683  3552 leader_election.cc:304] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:01.071919  4087 raft_consensus.cc:499] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:01.071976  4030 raft_consensus.cc:2826] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:01.072077  4030 raft_consensus.cc:499] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:01.072219  4030 raft_consensus.cc:3096] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.072235  4087 raft_consensus.cc:521] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.073030  4087 leader_election.cc:290] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:01.073980  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9a19cb906c974009aa9df99a14a78024" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:01.074193  3902 raft_consensus.cc:2490] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:01.074751  4017 leader_election.cc:290] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:01.075047  4030 raft_consensus.cc:521] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.076458  3269 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_bca0059ea2004174b89866f3e8cc68fb" } modify_external_catalogs: true
I20261118 02:55:01.076751  3269 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_bca0059ea2004174b89866f3e8cc68fb" } modify_external_catalogs: true
I20261118 02:55:01.077927  4030 leader_election.cc:290] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:01.078564  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9a19cb906c974009aa9df99a14a78024" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:01.078768  3902 raft_consensus.cc:3096] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.081962  3902 raft_consensus.cc:2490] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:01.083328  3553 leader_election.cc:304] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:01.084998  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "9a19cb906c974009aa9df99a14a78024" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:01.085238  3755 raft_consensus.cc:3096] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.088471  3755 raft_consensus.cc:2490] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:01.089386  4030 raft_consensus.cc:2826] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:01.089607  4030 raft_consensus.cc:705] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:01.089985  4030 consensus_queue.cc:238] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.104441  4017 consensus_queue.cc:1049] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.109282  4017 consensus_queue.cc:1049] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 02:55:01.120455  3838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.229.65:33705, user_credentials={real_user=slave}} blocked reactor thread for 105128us
I20261118 02:55:01.121537  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8f853e6b923444efa79cfbab2e31540e" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:01.121726  3618 raft_consensus.cc:2490] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:01.122246  4058 raft_consensus.cc:499] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:01.122401  4058 raft_consensus.cc:521] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.123129  4058 leader_election.cc:290] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:01.123740  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "55f81e7b62e54ec18c7c13a200edb68f" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:01.123821  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "55f81e7b62e54ec18c7c13a200edb68f" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:01.124086  3618 raft_consensus.cc:2415] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 99a39b1d9d814deeb3a8a644e9b392ef in current term 1: Already voted for candidate 3f1973986ec1466eab5bec01dd81654f in this term.
I20261118 02:55:01.124585  3838 leader_election.cc:304] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:01.124856  4083 consensus_queue.cc:1049] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.125450  4083 raft_consensus.cc:2826] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:01.125617  4083 raft_consensus.cc:499] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:01.125764  4083 raft_consensus.cc:3096] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.129177  4083 raft_consensus.cc:521] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.129601  3691 leader_election.cc:304] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f
I20261118 02:55:01.129951  4058 raft_consensus.cc:3096] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.133100  4058 raft_consensus.cc:2771] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 02:55:01.134994  4012 tablet_bootstrap.cc:492] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:01.135224  4012 ts_tablet_manager.cc:1449] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.180s	user 0.019s	sys 0.103s
I20261118 02:55:01.136164  4012 raft_consensus.cc:360] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.136442  4012 raft_consensus.cc:386] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:01.136569  4012 raft_consensus.cc:748] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:01.136917  4012 consensus_queue.cc:261] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.137537  4012 ts_tablet_manager.cc:1480] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20261118 02:55:01.137873  4012 tablet_bootstrap.cc:492] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:01.139792  4012 tablet_bootstrap.cc:654] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:01.150574  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "55f81e7b62e54ec18c7c13a200edb68f" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:01.150856  3755 raft_consensus.cc:2490] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:01.151506  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "55f81e7b62e54ec18c7c13a200edb68f" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:01.154713  3755 raft_consensus.cc:2490] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:01.156157  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8f853e6b923444efa79cfbab2e31540e" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:01.156404  3755 raft_consensus.cc:2490] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:01.157614  4083 leader_election.cc:290] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:01.140549  4011 ts_tablet_manager.cc:1480] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.098s	user 0.002s	sys 0.001s
I20261118 02:55:01.159271  4013 tablet_bootstrap.cc:492] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:01.159497  4083 consensus_queue.cc:1049] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.160163  4013 ts_tablet_manager.cc:1449] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.109s	user 0.004s	sys 0.000s
I20261118 02:55:01.161015  4013 raft_consensus.cc:360] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.158180  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8f853e6b923444efa79cfbab2e31540e" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:01.161988  3617 raft_consensus.cc:3096] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.162770  4013 raft_consensus.cc:386] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:01.162910  4013 raft_consensus.cc:748] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:01.163129  4013 consensus_queue.cc:261] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.164708  3617 raft_consensus.cc:2490] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:01.165182  3838 leader_election.cc:304] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:01.165723  4083 raft_consensus.cc:2826] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:01.166100  4083 raft_consensus.cc:705] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:01.166388  4083 consensus_queue.cc:238] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.168346  4013 ts_tablet_manager.cc:1480] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.000s
I20261118 02:55:01.169687  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8f853e6b923444efa79cfbab2e31540e" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:01.169888  3755 raft_consensus.cc:3096] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.174764  3755 raft_consensus.cc:2490] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
W20261118 02:55:01.181120  3263 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.229.124:36695, user_credentials={real_user=slave}} blocked reactor thread for 208234us
I20261118 02:55:01.186060  3881 tablet_service.cc:1500] Processing CreateTablet for tablet 5f05a8e5b21945a5bd3df86a5467a615 (DEFAULT_TABLE table=default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:01.186553  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f05a8e5b21945a5bd3df86a5467a615. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:01.193660  4011 tablet_bootstrap.cc:492] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:01.195525  4011 tablet_bootstrap.cc:654] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:01.213032  4011 tablet_bootstrap.cc:492] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:01.213302  4011 ts_tablet_manager.cc:1449] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.020s	user 0.001s	sys 0.008s
I20261118 02:55:01.213562  4030 raft_consensus.cc:499] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:01.213743  4030 raft_consensus.cc:521] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.214416  4030 leader_election.cc:290] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:01.215081  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "6bf520b76802491da8e3b090ad2152da" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:01.215914  3902 raft_consensus.cc:2490] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:01.216032  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "6bf520b76802491da8e3b090ad2152da" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:01.216260  3755 raft_consensus.cc:2490] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:01.216509  3553 leader_election.cc:304] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:01.217114  4030 raft_consensus.cc:2826] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:01.217226  4030 raft_consensus.cc:499] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:01.217320  4030 raft_consensus.cc:3096] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.220525  4030 raft_consensus.cc:521] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.222060  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "6bf520b76802491da8e3b090ad2152da" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:01.222258  3902 raft_consensus.cc:3096] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.225169  3902 raft_consensus.cc:2490] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:01.225698  3553 leader_election.cc:304] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:01.226019  4017 raft_consensus.cc:2826] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:01.226150  4017 raft_consensus.cc:705] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:01.226444  4017 consensus_queue.cc:238] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.229610  4030 leader_election.cc:290] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:01.231071  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "6bf520b76802491da8e3b090ad2152da" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:01.231716  3755 raft_consensus.cc:3096] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.232752  4011 raft_consensus.cc:360] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.235993  3755 raft_consensus.cc:2490] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:01.241838  4017 consensus_queue.cc:1049] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.244262  4011 raft_consensus.cc:386] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:01.244421  4011 raft_consensus.cc:748] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:01.244746  4011 consensus_queue.cc:261] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.246165  4011 ts_tablet_manager.cc:1480] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.000s
I20261118 02:55:01.249260  4012 tablet_bootstrap.cc:492] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:01.249439  4012 ts_tablet_manager.cc:1449] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.112s	user 0.010s	sys 0.016s
I20261118 02:55:01.250572  4012 raft_consensus.cc:360] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.250659  4030 consensus_queue.cc:1049] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.251631  4012 raft_consensus.cc:386] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:01.251731  4012 raft_consensus.cc:748] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:01.251968  4012 consensus_queue.cc:261] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.254069  4012 ts_tablet_manager.cc:1480] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 02:55:01.261523  4019 consensus_queue.cc:1049] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.273392  4019 consensus_queue.cc:1049] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.286320  4019 raft_consensus.cc:499] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:01.286552  4019 raft_consensus.cc:521] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.287194  4019 leader_election.cc:290] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:01.288276  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5f05a8e5b21945a5bd3df86a5467a615" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:01.288504  3617 raft_consensus.cc:2490] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:01.290966  3838 leader_election.cc:304] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:01.291399  4087 raft_consensus.cc:2826] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:01.291510  4087 raft_consensus.cc:499] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:01.291585  4087 raft_consensus.cc:3096] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.294031  4087 raft_consensus.cc:521] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.294570  4087 leader_election.cc:290] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:01.299003  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5f05a8e5b21945a5bd3df86a5467a615" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:01.299207  3752 raft_consensus.cc:3096] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.301959  3752 raft_consensus.cc:2490] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:01.305069  4058 raft_consensus.cc:499] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:01.305225  4058 raft_consensus.cc:521] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.305819  4058 leader_election.cc:290] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:01.306428  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ffe8d4c111d14054b76f9c7724072a94" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:01.306638  3902 raft_consensus.cc:2490] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:01.307621  3690 leader_election.cc:304] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:01.307973  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ffe8d4c111d14054b76f9c7724072a94" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:01.308045  3836 leader_election.cc:304] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:01.308195  3617 raft_consensus.cc:2490] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:01.308547  4083 raft_consensus.cc:2826] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:01.308665  4083 raft_consensus.cc:705] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:01.308970  4083 consensus_queue.cc:238] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:01.309405  4058 raft_consensus.cc:2826] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:01.309492  4058 raft_consensus.cc:499] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:01.309582  4058 raft_consensus.cc:3096] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.302659  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5f05a8e5b21945a5bd3df86a5467a615" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:01.313270  3755 raft_consensus.cc:2398] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1. Re-sending same reply.
I20261118 02:55:01.315402  4058 raft_consensus.cc:521] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.316243  4058 leader_election.cc:290] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:01.316540  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ffe8d4c111d14054b76f9c7724072a94" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:01.316818  3902 raft_consensus.cc:3096] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.317504  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ffe8d4c111d14054b76f9c7724072a94" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:01.317711  3617 raft_consensus.cc:3096] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.320818  3617 raft_consensus.cc:2490] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:01.321260  3691 leader_election.cc:304] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:01.321542  4058 raft_consensus.cc:2826] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:01.321683  4058 raft_consensus.cc:705] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:01.321924  4058 consensus_queue.cc:238] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:01.325083  3902 raft_consensus.cc:2490] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:01.329159  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "5f05a8e5b21945a5bd3df86a5467a615" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:01.330018  3618 raft_consensus.cc:3096] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:01.333078  4083 consensus_queue.cc:1049] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.337407  3618 raft_consensus.cc:2490] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:01.347039  4083 consensus_queue.cc:1049] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.350419  4105 consensus_queue.cc:1049] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20261118 02:55:01.351373  4058 consensus_queue.cc:1049] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:01.592870  4110 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:55:01.593219  4110 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:01.594197  4110 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:01.594687  3287 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 4 candidate_status { last_received { term: 3 index: 36 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
W20261118 02:55:01.628887  4047 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:55:01.629298  4047 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:55:01.629408  4047 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:55:01.630364  3333 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:01.630635  3333 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:01.630990  4110 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20261118 02:55:01.643215  4047 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:55:01.643329  4047 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:55:01.643406  4047 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:55:01.643468  4047 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:55:01.660548  4047 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.124:36695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.229.124
--webserver_port=39557
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:55:01.660801  3277 catalog_manager.cc:3543] renamed table default.table_dc3979f8c8b148caa37b0a8ef420ba9c [id=052f3134115244a6a2443e96121313e6] in HMS: new name default.table_8ae376d4d4d64414b5c08dc3c2e58bc4
I20261118 02:55:01.662734  4047 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:55:01.665131  4047 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:55:01.677729  3271 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_87d592a09c3348028719a603956fbce6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:01.678263  3270 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_a14cb6414555446286bf35bc5c6fb910 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:01.677757  3273 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_22f31dd1ff9e4185a6618f11b5493434 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:01.679179  3268 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_2e3917ff98f2474984580e61caa9c17b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:01.691787  4117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:01.693372  4118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:01.698589  4047 server_base.cc:1061] running on GCE node
W20261118 02:55:01.699322  4121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:01.700428  4047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:55:01.716599  4047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:55:01.718187  4047 hybrid_clock.cc:648] HybridClock initialized: now 1794970501718094 us; error 95 us; skew 500 ppm
I20261118 02:55:01.718766  4047 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:55:01.752295  3277 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:52040 (request call id 11) took 1628 ms. Trace:
I20261118 02:55:01.752509  3277 rpcz_store.cc:276] 1118 02:55:00.123548 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.123601 (+    53us) service_pool.cc:224] Handling call
1118 02:55:01.064934 (+941333us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:01.751542 (+686608us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:01.751876 (+   334us) catalog_manager.cc:3766] Apply alter schema
1118 02:55:01.752251 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":7953,"Hive Metastore.run_cpu_time_us":1145,"Hive Metastore.run_wall_time_us":587486}
I20261118 02:55:01.753866  3277 catalog_manager.cc:5840] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: UNKNOWN } } }
I20261118 02:55:01.767570  3277 catalog_manager.cc:5840] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20261118 02:55:01.778620  4047 webserver.cc:577] Webserver started at http://127.2.229.124:39557/ using document root <none> and password file <none>
I20261118 02:55:01.779771  4047 fs_manager.cc:362] Metadata directory not provided
I20261118 02:55:01.779938  4047 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:55:01.796772  4047 fs_manager.cc:714] Time spent opening directory manager: real 0.014s	user 0.002s	sys 0.005s
I20261118 02:55:01.806308  4133 tablet.cc:1759] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:01.814383  4132 tablet.cc:1759] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:01.816519  4131 tablet.cc:1759] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:01.819792  4136 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:55:01.821978  4047 fs_manager.cc:730] Time spent opening block manager: real 0.021s	user 0.003s	sys 0.001s
I20261118 02:55:01.822213  4047 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
uuid: "813a6937cea442ef81645fccc32207f4"
format_stamp: "Formatted at 2026-11-18 02:54:57 on dist-test-slave-49p1"
I20261118 02:55:01.823239  4047 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20261118 02:55:01.835572  3263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111) [suppressed 11 similar messages]
I20261118 02:55:01.844447  3277 catalog_manager.cc:5840] T ffe8d4c111d14054b76f9c7724072a94 P 3f1973986ec1466eab5bec01dd81654f reported cstate change: term changed from 0 to 1, leader changed from <none> to 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66). New cstate: current_term: 1 leader_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } }
I20261118 02:55:01.845209  3277 catalog_manager.cc:5840] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: HEALTHY } } }
I20261118 02:55:01.845793  3277 catalog_manager.cc:5840] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: HEALTHY } } }
I20261118 02:55:01.846341  3277 catalog_manager.cc:5840] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } }
I20261118 02:55:01.860460  4047 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:55:01.862219  4047 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:55:01.862593  4047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:55:01.865468  3277 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6] alter complete (version 1)
I20261118 02:55:01.871469  3277 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_571d1360811c42dc815efb439b2b52e6"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:01.884214  3275 catalog_manager.cc:3543] renamed table default.table_295a3e7e0d1f4e9db7078f6e5a5b9da1 [id=44b390866720455b8d2714eae004383b] in HMS: new name default.table_30f082fbe850451b80d98df75ed1081a
I20261118 02:55:01.954710  4187 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.124:36695 every 8 connection(s)
I20261118 02:55:01.954723  4047 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.124:36695
I20261118 02:55:01.957198  4047 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
I20261118 02:55:01.964733  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 4047
I20261118 02:55:01.965569  3274 catalog_manager.cc:3543] renamed table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7] in HMS: new name default.table_4f7796d5d5d54def843b971e66d15ea6
I20261118 02:55:01.968478  4188 sys_catalog.cc:263] Verifying existing consensus state
I20261118 02:55:01.972883  4188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap starting.
I20261118 02:55:01.985036  4188 log.cc:826] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:02.005002  4188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:02.005785  4188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap complete.
I20261118 02:55:02.014441  4188 raft_consensus.cc:360] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:02.015486  4188 raft_consensus.cc:748] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813a6937cea442ef81645fccc32207f4, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.016491  4188 consensus_queue.cc:261] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:02.018721  4192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:02.019734  4192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:02.021673  4188 sys_catalog.cc:565] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:55:02.045423  3272 catalog_manager.cc:3543] renamed table default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da] in HMS: new name default.table_a7f3710d39fb447da94e16d4ab3933fd
I20261118 02:55:02.052615  4207 catalog_manager.cc:820] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20261118 02:55:02.097942  3271 catalog_manager.cc:2215] created HMS catalog entry for table  [id=9724ed64fa5043bcbfee1caefac8e86d]
I20261118 02:55:02.104640  3271 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 22) took 1660 ms. Trace:
I20261118 02:55:02.104787  3271 rpcz_store.cc:276] 1118 02:55:00.444349 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.767401 (+323052us) service_pool.cc:224] Handling call
1118 02:55:01.677904 (+910503us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:01.678444 (+   540us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:02.097894 (+419450us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:02.104342 (+  6448us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:02.104455 (+   113us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:02.104570 (+   115us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:02.098761 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:02.098904 (+   143us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:02.098911 (+     7us) write_op.cc:435] Acquired schema lock
1118 02:55:02.098914 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:02.099018 (+   104us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:02.099033 (+    15us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:02.099040 (+     7us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:02.099138 (+    98us) tablet.cc:709] Row locks acquired
1118 02:55:02.099140 (+     2us) write_op.cc:260] PREPARE: finished
1118 02:55:02.099183 (+    43us) write_op.cc:270] Start()
1118 02:55:02.099220 (+    37us) write_op.cc:276] Timestamp: P: 1794970502099070 usec, L: 0
1118 02:55:02.099224 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:02.099443 (+   219us) log.cc:844] Serialized 432 byte log entry
1118 02:55:02.103542 (+  4099us) op_driver.cc:466] REPLICATION: finished
1118 02:55:02.103675 (+   133us) write_op.cc:301] APPLY: starting
1118 02:55:02.103692 (+    17us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:02.103738 (+    46us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:02.103778 (+    40us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:02.103899 (+   121us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:02.103933 (+    34us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:02.103941 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:02.104021 (+    80us) log.cc:844] Serialized 28 byte log entry
1118 02:55:02.104087 (+    66us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.104142 (+    55us) write_op.cc:454] Released schema lock
1118 02:55:02.104149 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:02.104225 (+    76us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.104229 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":366798,"Hive Metastore.run_cpu_time_us":356,"Hive Metastore.run_wall_time_us":52256,"child_traces":[["op",{"apply.queue_time_us":85,"apply.run_cpu_time_us":540,"apply.run_wall_time_us":539,"num_ops":2,"prepare.queue_time_us":76,"prepare.run_cpu_time_us":810,"prepare.run_wall_time_us":811,"raft.queue_time_us":1095,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":320,"replication_time_us":4288}]]}
I20261118 02:55:02.105614  3271 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_a4f08da750c54c49b9b7d73ebcf52624" } modify_external_catalogs: true
I20261118 02:55:02.105814  3271 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_a4f08da750c54c49b9b7d73ebcf52624" } modify_external_catalogs: true
I20261118 02:55:02.118363  3598 tablet_service.cc:1500] Processing CreateTablet for tablet c92c29c30ab6411dabe6f02331d29b7a (DEFAULT_TABLE table=default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.118593  3881 tablet_service.cc:1500] Processing CreateTablet for tablet c92c29c30ab6411dabe6f02331d29b7a (DEFAULT_TABLE table=default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.118872  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92c29c30ab6411dabe6f02331d29b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.118957  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92c29c30ab6411dabe6f02331d29b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.125578  3734 tablet_service.cc:1500] Processing CreateTablet for tablet c92c29c30ab6411dabe6f02331d29b7a (DEFAULT_TABLE table=default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.126056  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c92c29c30ab6411dabe6f02331d29b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.149720  4213 tablet_bootstrap.cc:492] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:02.151823  4213 tablet_bootstrap.cc:654] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.153157  4212 tablet_bootstrap.cc:492] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:02.155045  4212 tablet_bootstrap.cc:654] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.165388  4213 tablet_bootstrap.cc:492] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:02.165678  4213 ts_tablet_manager.cc:1449] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.016s	user 0.004s	sys 0.000s
I20261118 02:55:02.166639  4213 raft_consensus.cc:360] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.167119  4213 raft_consensus.cc:386] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.167273  4213 raft_consensus.cc:748] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.167644  4213 consensus_queue.cc:261] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.168357  4213 ts_tablet_manager.cc:1480] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:02.175135  4212 tablet_bootstrap.cc:492] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:02.175374  4212 ts_tablet_manager.cc:1449] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.000s
I20261118 02:55:02.176163  4211 tablet_bootstrap.cc:492] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:02.176326  4212 raft_consensus.cc:360] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.176578  4212 raft_consensus.cc:386] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.176705  4212 raft_consensus.cc:748] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.177008  4212 consensus_queue.cc:261] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.177603  4212 ts_tablet_manager.cc:1480] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:02.179162  4211 tablet_bootstrap.cc:654] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.183950  3948 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:55:02.188684  4153 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" instance_seqno: 1794970499527172) as {username='slave'} at 127.2.229.67:52477; Asking this server to re-register.
I20261118 02:55:02.195184  3948 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:02.198149  4153 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:02.206529  3801 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:55:02.209568  4153 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" instance_seqno: 1794970499028671) as {username='slave'} at 127.2.229.66:50711; Asking this server to re-register.
I20261118 02:55:02.209733  4211 tablet_bootstrap.cc:492] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:02.209895  4211 ts_tablet_manager.cc:1449] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.013s
I20261118 02:55:02.210455  3801 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:02.212265  4153 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:02.213104  4211 raft_consensus.cc:360] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.213397  4211 raft_consensus.cc:386] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.213542  4211 raft_consensus.cc:748] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.215049  4211 consensus_queue.cc:261] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.219259  4211 ts_tablet_manager.cc:1480] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.000s
I20261118 02:55:02.224360  3664 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:55:02.226665  4153 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" instance_seqno: 1794970498532688) as {username='slave'} at 127.2.229.65:46439; Asking this server to re-register.
I20261118 02:55:02.236189  3664 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:02.238484  3276 catalog_manager.cc:3543] renamed table default.table_df2a47bb353f4cdcb4c16399a99cb722 [id=8a5ea4b95bee4db7af9ba875250d564b] in HMS: new name default.table_0385cef6593c419eb426657c8cbef2bf
I20261118 02:55:02.239441  4153 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:02.277581  3270 catalog_manager.cc:2215] created HMS catalog entry for table  [id=0ce698410f004edaa9ad4ba1afe2de73]
I20261118 02:55:02.283869  3270 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 21) took 1867 ms. Trace:
I20261118 02:55:02.284021  3270 rpcz_store.cc:276] 1118 02:55:00.416429 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.648272 (+231843us) service_pool.cc:224] Handling call
1118 02:55:01.678371 (+1030099us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:01.678780 (+   409us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:02.277527 (+598747us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:02.283561 (+  6034us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:02.283695 (+   134us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:02.283825 (+   130us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:02.278391 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:02.278506 (+   115us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:02.278512 (+     6us) write_op.cc:435] Acquired schema lock
1118 02:55:02.278515 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:02.278602 (+    87us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:02.278614 (+    12us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:02.278619 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:02.278694 (+    75us) tablet.cc:709] Row locks acquired
1118 02:55:02.278696 (+     2us) write_op.cc:260] PREPARE: finished
1118 02:55:02.278739 (+    43us) write_op.cc:270] Start()
1118 02:55:02.278771 (+    32us) write_op.cc:276] Timestamp: P: 1794970502278689 usec, L: 0
1118 02:55:02.278775 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:02.278967 (+   192us) log.cc:844] Serialized 432 byte log entry
1118 02:55:02.282695 (+  3728us) op_driver.cc:466] REPLICATION: finished
1118 02:55:02.282822 (+   127us) write_op.cc:301] APPLY: starting
1118 02:55:02.282840 (+    18us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:02.282881 (+    41us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:02.282907 (+    26us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:02.283006 (+    99us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:02.283039 (+    33us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:02.283046 (+     7us) write_op.cc:312] APPLY: finished
1118 02:55:02.283118 (+    72us) log.cc:844] Serialized 28 byte log entry
1118 02:55:02.283185 (+    67us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.283204 (+    19us) write_op.cc:454] Released schema lock
1118 02:55:02.283211 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:02.283280 (+    69us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.283283 (+     3us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":559564,"Hive Metastore.run_cpu_time_us":291,"Hive Metastore.run_wall_time_us":38786,"child_traces":[["op",{"apply.queue_time_us":77,"apply.run_cpu_time_us":451,"apply.run_wall_time_us":451,"num_ops":2,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":678,"prepare.run_wall_time_us":678,"raft.queue_time_us":489,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":271,"replication_time_us":3890}]]}
I20261118 02:55:02.295192  4192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:02.295473  4192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:02.296294  4126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:02.296555  4126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:02.311007  3598 tablet_service.cc:1500] Processing CreateTablet for tablet 65bd6fcaed5b47c3a2478d375ace9636 (DEFAULT_TABLE table=default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.311528  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65bd6fcaed5b47c3a2478d375ace9636. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.311496  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 65bd6fcaed5b47c3a2478d375ace9636 (DEFAULT_TABLE table=default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.311944  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65bd6fcaed5b47c3a2478d375ace9636. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.314611  3273 catalog_manager.cc:2215] created HMS catalog entry for table  [id=aea45564ce10469393d3c7cc03b0bb9d]
I20261118 02:55:02.315302  3881 tablet_service.cc:1500] Processing CreateTablet for tablet 65bd6fcaed5b47c3a2478d375ace9636 (DEFAULT_TABLE table=default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.315774  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65bd6fcaed5b47c3a2478d375ace9636. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.318720  4211 tablet_bootstrap.cc:492] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:02.320504  4211 tablet_bootstrap.cc:654] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.323490  3270 master_service.cc:965] ReplaceTablet: received request to replace tablet 6bf520b76802491da8e3b090ad2152da from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:02.329592  4212 tablet_bootstrap.cc:492] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:02.331429  4212 tablet_bootstrap.cc:654] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.333096  3273 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 14) took 2009 ms. Trace:
I20261118 02:55:02.333614  4213 tablet_bootstrap.cc:492] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:02.333204  3273 rpcz_store.cc:276] 1118 02:55:00.323551 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.349401 (+ 25850us) service_pool.cc:224] Handling call
1118 02:55:01.678955 (+1329554us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:01.679415 (+   460us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:02.314569 (+635154us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:02.332799 (+ 18230us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:02.332918 (+   119us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:02.333036 (+   118us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:02.316651 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:02.316811 (+   160us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:02.316817 (+     6us) write_op.cc:435] Acquired schema lock
1118 02:55:02.316820 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:02.316960 (+   140us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:02.316974 (+    14us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:02.316979 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:02.317060 (+    81us) tablet.cc:709] Row locks acquired
1118 02:55:02.317063 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:02.317103 (+    40us) write_op.cc:270] Start()
1118 02:55:02.317140 (+    37us) write_op.cc:276] Timestamp: P: 1794970502317046 usec, L: 0
1118 02:55:02.317144 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:02.317347 (+   203us) log.cc:844] Serialized 432 byte log entry
1118 02:55:02.329320 (+ 11973us) op_driver.cc:466] REPLICATION: finished
1118 02:55:02.332185 (+  2865us) write_op.cc:301] APPLY: starting
1118 02:55:02.332202 (+    17us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:02.332240 (+    38us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:02.332274 (+    34us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:02.332383 (+   109us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:02.332415 (+    32us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:02.332423 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:02.332495 (+    72us) log.cc:844] Serialized 28 byte log entry
1118 02:55:02.332555 (+    60us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.332573 (+    18us) write_op.cc:454] Released schema lock
1118 02:55:02.332580 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:02.332648 (+    68us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.332652 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":597749,"Hive Metastore.run_cpu_time_us":280,"Hive Metastore.run_wall_time_us":37187,"child_traces":[["op",{"apply.queue_time_us":2794,"apply.run_cpu_time_us":463,"apply.run_wall_time_us":464,"num_ops":2,"prepare.queue_time_us":1283,"prepare.run_cpu_time_us":781,"prepare.run_wall_time_us":781,"replication_time_us":12146}]]}
I20261118 02:55:02.335309  4213 tablet_bootstrap.cc:654] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.343427  3273 master_service.cc:965] ReplaceTablet: received request to replace tablet 4ea7c6ad69d94b06bb0e72468db5c834 from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:02.352924  3270 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet 6bf520b76802491da8e3b090ad2152da
I20261118 02:55:02.354514  3270 catalog_manager.cc:6936] ReplaceTablet: tablet 6bf520b76802491da8e3b090ad2152da deleted and replaced by tablet 2de409a48f5b488484d74008f186d166
I20261118 02:55:02.354751  3270 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:52040 (request call id 28) took 1612 ms. Trace:
I20261118 02:55:02.355038  3598 tablet_service.cc:1547] Processing DeleteTablet for tablet 6bf520b76802491da8e3b090ad2152da with delete_type TABLET_DATA_DELETED (replaced by tablet 2de409a48f5b488484d74008f186d166) from {username='slave'} at 127.0.0.1:52990
I20261118 02:55:02.355044  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet 6bf520b76802491da8e3b090ad2152da with delete_type TABLET_DATA_DELETED (replaced by tablet 2de409a48f5b488484d74008f186d166) from {username='slave'} at 127.0.0.1:50234
I20261118 02:55:02.355412  3270 rpcz_store.cc:276] 1118 02:55:00.742530 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:02.323403 (+1580873us) service_pool.cc:224] Handling call
1118 02:55:02.354695 (+ 31292us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:02.338524 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:02.338693 (+   169us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:02.338700 (+     7us) write_op.cc:435] Acquired schema lock
1118 02:55:02.338703 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:02.338833 (+   130us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:02.338848 (+    15us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:02.338854 (+     6us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:02.338939 (+    85us) tablet.cc:709] Row locks acquired
1118 02:55:02.338942 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:02.338985 (+    43us) write_op.cc:270] Start()
1118 02:55:02.339024 (+    39us) write_op.cc:276] Timestamp: P: 1794970502338925 usec, L: 0
1118 02:55:02.339028 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:02.339278 (+   250us) log.cc:844] Serialized 658 byte log entry
1118 02:55:02.351807 (+ 12529us) op_driver.cc:466] REPLICATION: finished
1118 02:55:02.351905 (+    98us) write_op.cc:301] APPLY: starting
1118 02:55:02.351921 (+    16us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:02.351959 (+    38us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:02.351987 (+    28us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:02.352083 (+    96us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:02.352161 (+    78us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 02:55:02.352170 (+     9us) write_op.cc:312] APPLY: finished
1118 02:55:02.352241 (+    71us) log.cc:844] Serialized 28 byte log entry
1118 02:55:02.352309 (+    68us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.352329 (+    20us) write_op.cc:454] Released schema lock
1118 02:55:02.352336 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:02.352405 (+    69us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.352409 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":51,"apply.run_cpu_time_us":491,"apply.run_wall_time_us":490,"num_ops":2,"prepare.queue_time_us":4341,"prepare.run_cpu_time_us":843,"prepare.run_wall_time_us":843,"replication_time_us":12745}]]}
I20261118 02:55:02.356783  4229 tablet_replica.cc:333] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:02.359972  4229 raft_consensus.cc:2261] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:02.360608  4229 raft_consensus.cc:2290] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.360976  3734 tablet_service.cc:1547] Processing DeleteTablet for tablet 6bf520b76802491da8e3b090ad2152da with delete_type TABLET_DATA_DELETED (replaced by tablet 2de409a48f5b488484d74008f186d166) from {username='slave'} at 127.0.0.1:57880
I20261118 02:55:02.361258  4229 ts_tablet_manager.cc:1962] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.362175  3268 catalog_manager.cc:2215] created HMS catalog entry for table  [id=2584489d44ac4c6b90ca7c9f390c617e]
I20261118 02:55:02.367789  4230 tablet_replica.cc:333] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:02.368572  4230 raft_consensus.cc:2261] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:02.368898  3273 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet 4ea7c6ad69d94b06bb0e72468db5c834
I20261118 02:55:02.369431  4231 tablet_replica.cc:333] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:02.369632  3598 tablet_service.cc:1547] Processing DeleteTablet for tablet 4ea7c6ad69d94b06bb0e72468db5c834 with delete_type TABLET_DATA_DELETED (replaced by tablet e71c5cd71ebf4fa6bd158d59ed2b191d) from {username='slave'} at 127.0.0.1:52990
I20261118 02:55:02.370877  4231 raft_consensus.cc:2261] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:02.371163  4231 raft_consensus.cc:2290] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.371469  4230 raft_consensus.cc:2290] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.372083  4230 ts_tablet_manager.cc:1962] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.373778  3881 tablet_service.cc:1500] Processing CreateTablet for tablet a013d180f2cd430ab1ba30b39fd27591 (DEFAULT_TABLE table=default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.374286  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a013d180f2cd430ab1ba30b39fd27591. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.377398  3598 tablet_service.cc:1500] Processing CreateTablet for tablet a013d180f2cd430ab1ba30b39fd27591 (DEFAULT_TABLE table=default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.377952  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a013d180f2cd430ab1ba30b39fd27591. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.379402  4211 tablet_bootstrap.cc:492] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:02.384011  3268 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 24) took 1831 ms. Trace:
I20261118 02:55:02.384150  3268 rpcz_store.cc:276] 1118 02:55:00.552748 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.904912 (+352164us) service_pool.cc:224] Handling call
1118 02:55:01.679266 (+774354us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:01.679660 (+   394us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:02.362137 (+682477us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:02.383726 (+ 21589us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:02.383838 (+   112us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:02.383940 (+   102us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:02.368143 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:02.368266 (+   123us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:02.368271 (+     5us) write_op.cc:435] Acquired schema lock
1118 02:55:02.368273 (+     2us) tablet.cc:670] Decoding operations
1118 02:55:02.368354 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:02.368363 (+     9us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:02.368368 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:02.368433 (+    65us) tablet.cc:709] Row locks acquired
1118 02:55:02.368436 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:02.368464 (+    28us) write_op.cc:270] Start()
1118 02:55:02.368497 (+    33us) write_op.cc:276] Timestamp: P: 1794970502368402 usec, L: 0
1118 02:55:02.368500 (+     3us) op_driver.cc:350] REPLICATION: starting
1118 02:55:02.378467 (+  9967us) log.cc:844] Serialized 432 byte log entry
1118 02:55:02.378752 (+   285us) spinlock_profiling.cc:245] Waited 9.57 ms on lock 0x616000043354. stack: 00007f9c4434a3a6 00007f9c4434a160 00007f9c4333b34d 00007f9c5a90e704 00007f9c5a90e6cd 00007f9c5a90e09a 00007f9c5039c632 00007f9c517c0935 00007f9c517bf459 00007f9c517c7880 00007f9c517c768d 0000000000372c7c 00007f9c4439283f 00007f9c44395b3b 00007f9c443959b8 0000000000372c7c
1118 02:55:02.382673 (+  3921us) op_driver.cc:466] REPLICATION: finished
1118 02:55:02.383283 (+   610us) write_op.cc:301] APPLY: starting
1118 02:55:02.383299 (+    16us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:02.383341 (+    42us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:02.383344 (+     3us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:02.383448 (+   104us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:02.383480 (+    32us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:02.383487 (+     7us) write_op.cc:312] APPLY: finished
1118 02:55:02.383559 (+    72us) log.cc:844] Serialized 28 byte log entry
1118 02:55:02.383622 (+    63us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.383641 (+    19us) write_op.cc:454] Released schema lock
1118 02:55:02.383648 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:02.383722 (+    74us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.383726 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":634760,"Hive Metastore.run_cpu_time_us":274,"Hive Metastore.run_wall_time_us":47388,"child_traces":[["op",{"apply.queue_time_us":548,"apply.run_cpu_time_us":437,"apply.run_wall_time_us":437,"num_ops":2,"prepare.queue_time_us":900,"prepare.run_cpu_time_us":1105,"prepare.run_wall_time_us":10678,"raft.queue_time_us":779,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":298,"replication_time_us":14144,"spinlock_wait_cycles":22015360}]]}
I20261118 02:55:02.384773  3268 master_service.cc:965] ReplaceTablet: received request to replace tablet 4c230b673b8f487fbc79bdfb11a48628 from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:02.386333  3734 tablet_service.cc:1500] Processing CreateTablet for tablet a013d180f2cd430ab1ba30b39fd27591 (DEFAULT_TABLE table=default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.393581  4211 ts_tablet_manager.cc:1449] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.005s
I20261118 02:55:02.394498  4211 raft_consensus.cc:360] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.394727  4211 raft_consensus.cc:386] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.394809  4211 raft_consensus.cc:748] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.396087  4230 ts_tablet_manager.cc:1975] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.396301  4230 log.cc:1199] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/6bf520b76802491da8e3b090ad2152da
I20261118 02:55:02.396831  4230 ts_tablet_manager.cc:1996] T 6bf520b76802491da8e3b090ad2152da P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:02.398039  3263 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet 6bf520b76802491da8e3b090ad2152da (table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
I20261118 02:55:02.400060  3268 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet 4c230b673b8f487fbc79bdfb11a48628
I20261118 02:55:02.400735  3268 catalog_manager.cc:6936] ReplaceTablet: tablet 4c230b673b8f487fbc79bdfb11a48628 deleted and replaced by tablet 78189cccd457466bb5d01231038fec24
I20261118 02:55:02.400923  3268 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:52040 (request call id 31) took 1542 ms. Trace:
I20261118 02:55:02.401161  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet 4c230b673b8f487fbc79bdfb11a48628 with delete_type TABLET_DATA_DELETED (replaced by tablet 78189cccd457466bb5d01231038fec24) from {username='slave'} at 127.0.0.1:50234
I20261118 02:55:02.401871  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet 4c230b673b8f487fbc79bdfb11a48628 with delete_type TABLET_DATA_DELETED (replaced by tablet 78189cccd457466bb5d01231038fec24) from {username='slave'} at 127.0.0.1:52990
I20261118 02:55:02.401073  3268 rpcz_store.cc:276] 1118 02:55:00.858542 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:02.384690 (+1526148us) service_pool.cc:224] Handling call
1118 02:55:02.392370 (+  7680us) spinlock_profiling.cc:245] Waited 7.29 ms on lock 0x616000043354. stack: 00007f9c4434a3a6 00007f9c4434a160 00007f9c4333b34d 00007f9c5a90e704 00007f9c5a90e6cd 00007f9c5386082d 00007f9c53860923 00007f9c503ca221 00007f9c582bf50d 00007f9c5845b6fb 00007f9c4ed63e67 00007f9c4ed63b95 00007f9c4b4928de 00007f9c4b4917be 00007f9c4b4979f8 00007f9c4b49835f
1118 02:55:02.400892 (+  8522us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:02.395389 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:02.395504 (+   115us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:02.395509 (+     5us) write_op.cc:435] Acquired schema lock
1118 02:55:02.395511 (+     2us) tablet.cc:670] Decoding operations
1118 02:55:02.395609 (+    98us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:02.395634 (+    25us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:02.395639 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:02.395711 (+    72us) tablet.cc:709] Row locks acquired
1118 02:55:02.395714 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:02.395752 (+    38us) write_op.cc:270] Start()
1118 02:55:02.395782 (+    30us) write_op.cc:276] Timestamp: P: 1794970502395682 usec, L: 0
1118 02:55:02.395786 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:02.396010 (+   224us) log.cc:844] Serialized 658 byte log entry
1118 02:55:02.398940 (+  2930us) op_driver.cc:466] REPLICATION: finished
1118 02:55:02.399075 (+   135us) write_op.cc:301] APPLY: starting
1118 02:55:02.399094 (+    19us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:02.399142 (+    48us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:02.399210 (+    68us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:02.399346 (+   136us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:02.399401 (+    55us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 02:55:02.399409 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:02.399480 (+    71us) log.cc:844] Serialized 28 byte log entry
1118 02:55:02.399726 (+   246us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.399746 (+    20us) write_op.cc:454] Released schema lock
1118 02:55:02.399753 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:02.399819 (+    66us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.399823 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":16767360,"child_traces":[["op",{"apply.queue_time_us":38,"apply.run_cpu_time_us":582,"apply.run_wall_time_us":745,"num_ops":2,"prepare.queue_time_us":2070,"prepare.run_cpu_time_us":684,"prepare.run_wall_time_us":722,"raft.queue_time_us":354,"replication_time_us":3124}]]}
I20261118 02:55:02.402199  4230 tablet_replica.cc:333] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:02.402442  4230 raft_consensus.cc:2261] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:02.402681  4230 raft_consensus.cc:2290] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.403153  3268 master_service.cc:965] ReplaceTablet: received request to replace tablet 4c230b673b8f487fbc79bdfb11a48628 from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:02.403334  4230 ts_tablet_manager.cc:1962] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.403766  3268 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:52040 (request call id 32) took 1543 ms. Trace:
I20261118 02:55:02.403903  3268 rpcz_store.cc:276] 1118 02:55:00.860437 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:02.403080 (+1542643us) service_pool.cc:224] Handling call
1118 02:55:02.403713 (+   633us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 02:55:02.404774  3268 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_c113c2b57e17479e92e241234a241af7" } modify_external_catalogs: true
I20261118 02:55:02.405263  3268 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_c113c2b57e17479e92e241234a241af7" } modify_external_catalogs: true
I20261118 02:55:02.405880  4211 consensus_queue.cc:261] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.406497  4211 ts_tablet_manager.cc:1480] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.001s
I20261118 02:55:02.409996  4230 ts_tablet_manager.cc:1975] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.410462  3270 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52040:
name: "default.Table_51d36ff687ca47b5995e5693d7b8931b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:02.371889  4231 ts_tablet_manager.cc:1962] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.412957  3273 catalog_manager.cc:6936] ReplaceTablet: tablet 4ea7c6ad69d94b06bb0e72468db5c834 deleted and replaced by tablet e71c5cd71ebf4fa6bd158d59ed2b191d
I20261118 02:55:02.413075  3273 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:52040 (request call id 29) took 1639 ms. Trace:
I20261118 02:55:02.413159  3273 rpcz_store.cc:276] 1118 02:55:00.773826 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:02.343341 (+1569515us) service_pool.cc:224] Handling call
1118 02:55:02.368602 (+ 25261us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.368608 (+     6us) write_op.cc:454] Released schema lock
1118 02:55:02.413044 (+ 44436us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:02.361862 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:02.361976 (+   114us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:02.361981 (+     5us) write_op.cc:435] Acquired schema lock
1118 02:55:02.361984 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:02.362084 (+   100us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:02.362095 (+    11us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:02.362100 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:02.362169 (+    69us) tablet.cc:709] Row locks acquired
1118 02:55:02.362172 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:02.362207 (+    35us) write_op.cc:270] Start()
1118 02:55:02.362237 (+    30us) write_op.cc:276] Timestamp: P: 1794970502362144 usec, L: 0
1118 02:55:02.362241 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:02.362420 (+   179us) log.cc:844] Serialized 658 byte log entry
1118 02:55:02.365341 (+  2921us) op_driver.cc:466] REPLICATION: finished
1118 02:55:02.365435 (+    94us) write_op.cc:301] APPLY: starting
1118 02:55:02.365447 (+    12us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:02.365491 (+    44us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:02.365516 (+    25us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:02.365605 (+    89us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:02.365667 (+    62us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 02:55:02.365674 (+     7us) write_op.cc:312] APPLY: finished
1118 02:55:02.365737 (+    63us) log.cc:844] Serialized 28 byte log entry
1118 02:55:02.365795 (+    58us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.365811 (+    16us) write_op.cc:454] Released schema lock
1118 02:55:02.365817 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":50,"apply.run_cpu_time_us":431,"apply.run_wall_time_us":430,"num_ops":2,"prepare.queue_time_us":17231,"prepare.run_cpu_time_us":663,"prepare.run_wall_time_us":663,"raft.queue_time_us":1191,"raft.run_cpu_time_us":563,"raft.run_wall_time_us":564,"replication_time_us":3075}]]}
I20261118 02:55:02.414131  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a013d180f2cd430ab1ba30b39fd27591. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.408886  3733 tablet_service.cc:1547] Processing DeleteTablet for tablet 4c230b673b8f487fbc79bdfb11a48628 with delete_type TABLET_DATA_DELETED (replaced by tablet 78189cccd457466bb5d01231038fec24) from {username='slave'} at 127.0.0.1:57880
I20261118 02:55:02.412323  3735 tablet_service.cc:1547] Processing DeleteTablet for tablet 4ea7c6ad69d94b06bb0e72468db5c834 with delete_type TABLET_DATA_DELETED (replaced by tablet e71c5cd71ebf4fa6bd158d59ed2b191d) from {username='slave'} at 127.0.0.1:57880
I20261118 02:55:02.417141  3597 tablet_service.cc:1500] Processing CreateTablet for tablet 2de409a48f5b488484d74008f186d166 (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.411877  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet 4ea7c6ad69d94b06bb0e72468db5c834 with delete_type TABLET_DATA_DELETED (replaced by tablet e71c5cd71ebf4fa6bd158d59ed2b191d) from {username='slave'} at 127.0.0.1:50234
I20261118 02:55:02.417624  4230 log.cc:1199] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/4c230b673b8f487fbc79bdfb11a48628
I20261118 02:55:02.418151  4230 ts_tablet_manager.cc:1996] T 4c230b673b8f487fbc79bdfb11a48628 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:02.420329  3735 tablet_service.cc:1500] Processing CreateTablet for tablet 2de409a48f5b488484d74008f186d166 (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.420812  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2de409a48f5b488484d74008f186d166. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.421273  3881 tablet_service.cc:1500] Processing CreateTablet for tablet 2de409a48f5b488484d74008f186d166 (DEFAULT_TABLE table=default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.423233  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2de409a48f5b488484d74008f186d166. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.423996  4229 ts_tablet_manager.cc:1975] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.424155  4229 log.cc:1199] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/6bf520b76802491da8e3b090ad2152da
I20261118 02:55:02.424561  4229 ts_tablet_manager.cc:1996] T 6bf520b76802491da8e3b090ad2152da P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:02.427963  4231 ts_tablet_manager.cc:1975] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.428192  4231 log.cc:1199] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/6bf520b76802491da8e3b090ad2152da
I20261118 02:55:02.428731  4231 ts_tablet_manager.cc:1996] T 6bf520b76802491da8e3b090ad2152da P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:02.429561  3596 tablet_service.cc:1500] Processing CreateTablet for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (DEFAULT_TABLE table=default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.429872  3262 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet 6bf520b76802491da8e3b090ad2152da (table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
I20261118 02:55:02.430226  4231 tablet_replica.cc:333] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:02.431022  3882 tablet_service.cc:1500] Processing CreateTablet for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (DEFAULT_TABLE table=default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.431501  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71c5cd71ebf4fa6bd158d59ed2b191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.431843  4231 raft_consensus.cc:2261] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:02.432245  3273 master_service.cc:965] ReplaceTablet: received request to replace tablet 9a19cb906c974009aa9df99a14a78024 from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:02.432397  4231 raft_consensus.cc:2290] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.432947  4231 ts_tablet_manager.cc:1962] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.431545  3735 tablet_service.cc:1500] Processing CreateTablet for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (DEFAULT_TABLE table=default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.435521  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71c5cd71ebf4fa6bd158d59ed2b191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.439710  4213 tablet_bootstrap.cc:492] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:02.439857  3273 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet 9a19cb906c974009aa9df99a14a78024
I20261118 02:55:02.439952  4213 ts_tablet_manager.cc:1449] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.106s	user 0.008s	sys 0.015s
I20261118 02:55:02.440919  3595 tablet_service.cc:1547] Processing DeleteTablet for tablet 9a19cb906c974009aa9df99a14a78024 with delete_type TABLET_DATA_DELETED (replaced by tablet e9323addc63245bfacb3854782425347) from {username='slave'} at 127.0.0.1:52990
I20261118 02:55:02.441011  4213 raft_consensus.cc:360] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.441272  4213 raft_consensus.cc:386] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.441354  4213 raft_consensus.cc:748] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.441845  3263 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet 4c230b673b8f487fbc79bdfb11a48628 (table default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]) successfully deleted
I20261118 02:55:02.442184  4230 tablet_replica.cc:333] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:02.442389  4213 consensus_queue.cc:261] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.442389  4230 raft_consensus.cc:2261] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:02.443663  3733 tablet_service.cc:1547] Processing DeleteTablet for tablet 9a19cb906c974009aa9df99a14a78024 with delete_type TABLET_DATA_DELETED (replaced by tablet e9323addc63245bfacb3854782425347) from {username='slave'} at 127.0.0.1:57880
I20261118 02:55:02.444335  4230 raft_consensus.cc:2290] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.444782  4213 ts_tablet_manager.cc:1480] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 02:55:02.445071  4213 tablet_bootstrap.cc:492] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:02.449146  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2de409a48f5b488484d74008f186d166. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.450366  4213 tablet_bootstrap.cc:654] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.452953  3596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71c5cd71ebf4fa6bd158d59ed2b191d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.461302  4229 tablet_replica.cc:333] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:02.461537  4229 raft_consensus.cc:2261] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:02.461993  4229 raft_consensus.cc:2290] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.462831  4229 ts_tablet_manager.cc:1962] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.463047  4213 tablet_bootstrap.cc:492] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:02.463263  4213 ts_tablet_manager.cc:1449] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.018s	user 0.002s	sys 0.004s
I20261118 02:55:02.464167  4213 raft_consensus.cc:360] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:02.464414  4213 raft_consensus.cc:386] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.464530  4213 raft_consensus.cc:748] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.464838  4213 consensus_queue.cc:261] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:02.466874  4213 ts_tablet_manager.cc:1480] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.003s	user 0.001s	sys 0.000s
I20261118 02:55:02.467221  4213 tablet_bootstrap.cc:492] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:02.468192  3264 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet 6bf520b76802491da8e3b090ad2152da (table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
I20261118 02:55:02.468778  4211 tablet_bootstrap.cc:492] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:02.470914  4213 tablet_bootstrap.cc:654] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.475835  4230 ts_tablet_manager.cc:1962] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.477862  4231 ts_tablet_manager.cc:1975] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.478031  4231 log.cc:1199] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/4c230b673b8f487fbc79bdfb11a48628
I20261118 02:55:02.479214  4213 tablet_bootstrap.cc:492] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:02.479290  4211 tablet_bootstrap.cc:654] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.479466  4213 ts_tablet_manager.cc:1449] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.012s	user 0.006s	sys 0.004s
I20261118 02:55:02.480432  4213 raft_consensus.cc:360] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.480701  4213 raft_consensus.cc:386] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.481022  4213 raft_consensus.cc:748] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.481406  4213 consensus_queue.cc:261] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.483502  4213 ts_tablet_manager.cc:1480] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 02:55:02.483843  4213 tablet_bootstrap.cc:492] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:02.487402  4231 ts_tablet_manager.cc:1996] T 4c230b673b8f487fbc79bdfb11a48628 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:02.487635  4213 tablet_bootstrap.cc:654] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.488178  4212 tablet_bootstrap.cc:492] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:02.488426  4212 ts_tablet_manager.cc:1449] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.159s	user 0.005s	sys 0.004s
I20261118 02:55:02.488587  3262 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet 4c230b673b8f487fbc79bdfb11a48628 (table default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]) successfully deleted
I20261118 02:55:02.490651  4212 raft_consensus.cc:360] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.490962  4212 raft_consensus.cc:386] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.491072  4212 raft_consensus.cc:748] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.491425  4212 consensus_queue.cc:261] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.492177  4212 ts_tablet_manager.cc:1480] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 02:55:02.492547  4212 tablet_bootstrap.cc:492] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:02.494400  4212 tablet_bootstrap.cc:654] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.497686  4229 ts_tablet_manager.cc:1975] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:02.499058  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet 9a19cb906c974009aa9df99a14a78024 with delete_type TABLET_DATA_DELETED (replaced by tablet e9323addc63245bfacb3854782425347) from {username='slave'} at 127.0.0.1:50234
I20261118 02:55:02.499738  4213 tablet_bootstrap.cc:492] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:02.499893  4213 ts_tablet_manager.cc:1449] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.016s	user 0.005s	sys 0.004s
I20261118 02:55:02.500203  3273 catalog_manager.cc:6936] ReplaceTablet: tablet 9a19cb906c974009aa9df99a14a78024 deleted and replaced by tablet e9323addc63245bfacb3854782425347
I20261118 02:55:02.500412  3273 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:52040 (request call id 34) took 1629 ms. Trace:
I20261118 02:55:02.500705  4213 raft_consensus.cc:360] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.500890  4213 raft_consensus.cc:386] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.500978  4213 raft_consensus.cc:748] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.501272  4213 consensus_queue.cc:261] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.502074  4229 log.cc:1199] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/4ea7c6ad69d94b06bb0e72468db5c834
I20261118 02:55:02.500546  3273 rpcz_store.cc:276] 1118 02:55:00.871173 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:02.432174 (+1561001us) service_pool.cc:224] Handling call
1118 02:55:02.500360 (+ 68186us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:02.434678 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:02.434809 (+   131us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:02.434816 (+     7us) write_op.cc:435] Acquired schema lock
1118 02:55:02.434819 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:02.434930 (+   111us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:02.434942 (+    12us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:02.434947 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:02.435016 (+    69us) tablet.cc:709] Row locks acquired
1118 02:55:02.435018 (+     2us) write_op.cc:260] PREPARE: finished
1118 02:55:02.435064 (+    46us) write_op.cc:270] Start()
1118 02:55:02.435099 (+    35us) write_op.cc:276] Timestamp: P: 1794970502434982 usec, L: 0
1118 02:55:02.435103 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:02.435386 (+   283us) log.cc:844] Serialized 658 byte log entry
1118 02:55:02.438512 (+  3126us) op_driver.cc:466] REPLICATION: finished
1118 02:55:02.438730 (+   218us) write_op.cc:301] APPLY: starting
1118 02:55:02.438747 (+    17us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:02.438794 (+    47us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:02.438819 (+    25us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:02.438915 (+    96us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:02.438972 (+    57us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 02:55:02.438980 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:02.439049 (+    69us) log.cc:844] Serialized 28 byte log entry
1118 02:55:02.439111 (+    62us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.439129 (+    18us) write_op.cc:454] Released schema lock
1118 02:55:02.439135 (+     6us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:02.439204 (+    69us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:02.439208 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":147,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":481,"num_ops":2,"prepare.queue_time_us":1327,"prepare.run_cpu_time_us":797,"prepare.run_wall_time_us":841,"raft.queue_time_us":1891,"raft.run_cpu_time_us":675,"raft.run_wall_time_us":1988,"replication_time_us":3377,"spinlock_wait_cycles":66048}]]}
I20261118 02:55:02.503659  4213 ts_tablet_manager.cc:1480] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 02:55:02.507390  4229 ts_tablet_manager.cc:1996] T 4ea7c6ad69d94b06bb0e72468db5c834 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:02.508754  4230 ts_tablet_manager.cc:1975] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:02.509831  4229 tablet_replica.cc:333] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:02.510353  3264 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet 4ea7c6ad69d94b06bb0e72468db5c834 (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]) successfully deleted
I20261118 02:55:02.494484  4231 tablet_replica.cc:333] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:02.510623  4230 log.cc:1199] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/4ea7c6ad69d94b06bb0e72468db5c834
I20261118 02:55:02.510756  4231 raft_consensus.cc:2261] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:02.511085  4231 raft_consensus.cc:2290] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.512200  4231 ts_tablet_manager.cc:1962] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.512450  4230 ts_tablet_manager.cc:1996] T 4ea7c6ad69d94b06bb0e72468db5c834 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:02.512244  4229 raft_consensus.cc:2261] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:02.513509  4230 tablet_replica.cc:333] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:02.513728  4229 raft_consensus.cc:2290] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.514289  4229 ts_tablet_manager.cc:1962] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.514379  3263 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet 4ea7c6ad69d94b06bb0e72468db5c834 (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]) successfully deleted
I20261118 02:55:02.515167  4230 raft_consensus.cc:2261] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:02.515419  4230 raft_consensus.cc:2290] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.517704  4230 ts_tablet_manager.cc:1962] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.518064  4215 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:02.518205  4215 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.518909  4215 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:02.520200  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c92c29c30ab6411dabe6f02331d29b7a" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:02.520404  3755 raft_consensus.cc:2490] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:02.521052  3836 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:02.521936  4215 raft_consensus.cc:2826] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:02.522042  4215 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:02.521989  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c92c29c30ab6411dabe6f02331d29b7a" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:02.522117  4215 raft_consensus.cc:3096] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.522203  3618 raft_consensus.cc:2490] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:02.523371  4212 tablet_bootstrap.cc:492] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:02.523517  4212 ts_tablet_manager.cc:1449] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.000s
I20261118 02:55:02.524334  4212 raft_consensus.cc:360] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.524521  4212 raft_consensus.cc:386] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.524616  4212 raft_consensus.cc:748] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.524766  4215 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.524919  4212 consensus_queue.cc:261] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.526166  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c92c29c30ab6411dabe6f02331d29b7a" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:02.526369  3618 raft_consensus.cc:3096] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.526746  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c92c29c30ab6411dabe6f02331d29b7a" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:02.526921  3755 raft_consensus.cc:3096] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.527249  4231 ts_tablet_manager.cc:1975] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:02.527379  4231 log.cc:1199] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/4ea7c6ad69d94b06bb0e72468db5c834
I20261118 02:55:02.527817  4231 ts_tablet_manager.cc:1996] T 4ea7c6ad69d94b06bb0e72468db5c834 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:02.528297  4212 ts_tablet_manager.cc:1480] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 02:55:02.528940  3262 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet 4ea7c6ad69d94b06bb0e72468db5c834 (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]) successfully deleted
I20261118 02:55:02.529165  3618 raft_consensus.cc:2490] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:02.530246  3755 raft_consensus.cc:2490] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:02.530700  3836 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:02.530949  4212 tablet_bootstrap.cc:492] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:02.531028  4210 raft_consensus.cc:2826] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:02.531577  4229 ts_tablet_manager.cc:1975] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.531693  4229 log.cc:1199] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/4c230b673b8f487fbc79bdfb11a48628
I20261118 02:55:02.532657  4231 tablet_replica.cc:333] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:02.532696  4229 ts_tablet_manager.cc:1996] T 4c230b673b8f487fbc79bdfb11a48628 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:02.532874  4231 raft_consensus.cc:2261] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:02.533051  4231 raft_consensus.cc:2290] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.533205  4212 tablet_bootstrap.cc:654] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.533658  4229 tablet_replica.cc:333] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:02.533648  4231 ts_tablet_manager.cc:1962] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.533855  4229 raft_consensus.cc:2261] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:02.534322  3264 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet 4c230b673b8f487fbc79bdfb11a48628 (table default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]) successfully deleted
I20261118 02:55:02.535019  4229 raft_consensus.cc:2290] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:02.536906  4215 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:02.536726  4229 ts_tablet_manager.cc:1962] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:02.501953  4211 tablet_bootstrap.cc:492] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:02.540230  4211 ts_tablet_manager.cc:1449] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.072s	user 0.001s	sys 0.003s
I20261118 02:55:02.540940  4211 raft_consensus.cc:360] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.541126  4211 raft_consensus.cc:386] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.541237  4211 raft_consensus.cc:748] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.541453  4211 consensus_queue.cc:261] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.546059  3273 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_287099a68a3540f1be0284c71d4d7056" } modify_external_catalogs: true
I20261118 02:55:02.548307  3273 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52040:
table { table_name: "default.Table_287099a68a3540f1be0284c71d4d7056" } modify_external_catalogs: true
I20261118 02:55:02.548563  4210 raft_consensus.cc:705] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:02.548883  4210 consensus_queue.cc:238] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.552723  3735 tablet_service.cc:1500] Processing CreateTablet for tablet 489beea71ae74236addee59c153cfee7 (DEFAULT_TABLE table=default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.553171  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489beea71ae74236addee59c153cfee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.553092  3881 tablet_service.cc:1500] Processing CreateTablet for tablet 489beea71ae74236addee59c153cfee7 (DEFAULT_TABLE table=default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.553480  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489beea71ae74236addee59c153cfee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.556941  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 78189cccd457466bb5d01231038fec24 (DEFAULT_TABLE table=default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.557351  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78189cccd457466bb5d01231038fec24. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.558972  3882 tablet_service.cc:1500] Processing CreateTablet for tablet 78189cccd457466bb5d01231038fec24 (DEFAULT_TABLE table=default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.559351  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78189cccd457466bb5d01231038fec24. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.559295  3596 tablet_service.cc:1500] Processing CreateTablet for tablet 78189cccd457466bb5d01231038fec24 (DEFAULT_TABLE table=default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.559686  3596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78189cccd457466bb5d01231038fec24. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.563303  4213 tablet_bootstrap.cc:492] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:02.565783  4213 tablet_bootstrap.cc:654] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.542007  4211 ts_tablet_manager.cc:1480] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 02:55:02.573877  4239 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:02.575912  4239 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:02.577234  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a013d180f2cd430ab1ba30b39fd27591" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
W20261118 02:55:02.578177  3836 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Illegal state: must be running to vote when last-logged opid is not known
I20261118 02:55:02.548470  4229 ts_tablet_manager.cc:1975] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.579119  4229 log.cc:1199] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/9a19cb906c974009aa9df99a14a78024
I20261118 02:55:02.552182  3598 tablet_service.cc:1500] Processing CreateTablet for tablet 489beea71ae74236addee59c153cfee7 (DEFAULT_TABLE table=default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.579300  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a013d180f2cd430ab1ba30b39fd27591" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:02.579530  4229 ts_tablet_manager.cc:1996] T 9a19cb906c974009aa9df99a14a78024 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:02.579659  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 489beea71ae74236addee59c153cfee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.583029  4231 ts_tablet_manager.cc:1975] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.583178  4231 log.cc:1199] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/9a19cb906c974009aa9df99a14a78024
I20261118 02:55:02.583501  4231 ts_tablet_manager.cc:1996] T 9a19cb906c974009aa9df99a14a78024 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
W20261118 02:55:02.583515  3838 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Illegal state: must be running to vote when last-logged opid is not known
I20261118 02:55:02.583633  3838 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:02.584122  3264 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet 9a19cb906c974009aa9df99a14a78024 (table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]) successfully deleted
I20261118 02:55:02.586706  4215 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 02:55:02.588336  4239 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:02.593148  4212 tablet_bootstrap.cc:492] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:02.593330  4212 ts_tablet_manager.cc:1449] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.005s
I20261118 02:55:02.594102  4212 raft_consensus.cc:360] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:02.594316  4212 raft_consensus.cc:386] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.594388  4212 raft_consensus.cc:748] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.594635  4212 consensus_queue.cc:261] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:02.596073  4212 ts_tablet_manager.cc:1480] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.001s
I20261118 02:55:02.596426  4212 tablet_bootstrap.cc:492] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:02.596057  3262 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet 9a19cb906c974009aa9df99a14a78024 (table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]) successfully deleted
I20261118 02:55:02.600968  4212 tablet_bootstrap.cc:654] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.616334  4211 tablet_bootstrap.cc:492] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:02.617105  4212 tablet_bootstrap.cc:492] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:02.617264  4212 ts_tablet_manager.cc:1449] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.001s
I20261118 02:55:02.618093  4212 raft_consensus.cc:360] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.618280  4212 raft_consensus.cc:386] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.618371  4212 raft_consensus.cc:748] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.618683  4212 consensus_queue.cc:261] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.620118  4212 ts_tablet_manager.cc:1480] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.001s
I20261118 02:55:02.620404  4212 tablet_bootstrap.cc:492] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:02.622658  4212 tablet_bootstrap.cc:654] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.627422  4211 tablet_bootstrap.cc:654] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.642204  3598 tablet_service.cc:1500] Processing CreateTablet for tablet e9323addc63245bfacb3854782425347 (DEFAULT_TABLE table=default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.642701  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9323addc63245bfacb3854782425347. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.645967  4210 consensus_queue.cc:1049] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:02.650708  4215 consensus_queue.cc:1049] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:02.667567  4213 tablet_bootstrap.cc:492] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:02.667779  4213 ts_tablet_manager.cc:1449] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.015s
I20261118 02:55:02.668637  4213 raft_consensus.cc:360] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.668875  4213 raft_consensus.cc:386] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.668968  4213 raft_consensus.cc:748] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.669234  4213 consensus_queue.cc:261] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.669760  4213 ts_tablet_manager.cc:1480] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.003s	sys 0.000s
I20261118 02:55:02.674525  3735 tablet_service.cc:1500] Processing CreateTablet for tablet e9323addc63245bfacb3854782425347 (DEFAULT_TABLE table=default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.675094  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9323addc63245bfacb3854782425347. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.677455  4213 tablet_bootstrap.cc:492] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:02.678975  3882 tablet_service.cc:1500] Processing CreateTablet for tablet e9323addc63245bfacb3854782425347 (DEFAULT_TABLE table=default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:02.679389  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9323addc63245bfacb3854782425347. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:02.680945  4213 tablet_bootstrap.cc:654] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.682927  4230 ts_tablet_manager.cc:1975] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:02.683077  4230 log.cc:1199] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/9a19cb906c974009aa9df99a14a78024
I20261118 02:55:02.683494  4230 ts_tablet_manager.cc:1996] T 9a19cb906c974009aa9df99a14a78024 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:02.684659  3263 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet 9a19cb906c974009aa9df99a14a78024 (table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]) successfully deleted
I20261118 02:55:02.694171  4212 tablet_bootstrap.cc:492] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:02.694375  4212 ts_tablet_manager.cc:1449] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.074s	user 0.027s	sys 0.023s
I20261118 02:55:02.695160  4212 raft_consensus.cc:360] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.695392  4212 raft_consensus.cc:386] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.695462  4212 raft_consensus.cc:748] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.695758  4212 consensus_queue.cc:261] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.696442  4212 ts_tablet_manager.cc:1480] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.002s
I20261118 02:55:02.697000  4212 tablet_bootstrap.cc:492] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:02.699144  4212 tablet_bootstrap.cc:654] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.718852  4211 tablet_bootstrap.cc:492] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:02.719098  4213 tablet_bootstrap.cc:492] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:02.719250  4213 ts_tablet_manager.cc:1449] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.043s	user 0.001s	sys 0.012s
I20261118 02:55:02.719424  4211 ts_tablet_manager.cc:1449] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.103s	user 0.003s	sys 0.007s
I20261118 02:55:02.720080  4213 raft_consensus.cc:360] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.720315  4213 raft_consensus.cc:386] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.720404  4213 raft_consensus.cc:748] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.720681  4213 consensus_queue.cc:261] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.721235  4213 ts_tablet_manager.cc:1480] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:02.720273  4211 raft_consensus.cc:360] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.722846  4213 tablet_bootstrap.cc:492] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:02.725155  4213 tablet_bootstrap.cc:654] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.733351  4215 raft_consensus.cc:499] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:02.733520  4215 raft_consensus.cc:521] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.734225  4215 leader_election.cc:290] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:02.734817  4211 raft_consensus.cc:386] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.734939  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "65bd6fcaed5b47c3a2478d375ace9636" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:02.735188  3617 raft_consensus.cc:2490] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:02.736142  3838 leader_election.cc:304] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:02.736533  4210 raft_consensus.cc:2826] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:02.736619  4210 raft_consensus.cc:499] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:02.736711  4210 raft_consensus.cc:3096] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.736858  4212 tablet_bootstrap.cc:492] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:02.737006  4212 ts_tablet_manager.cc:1449] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.040s	user 0.006s	sys 0.006s
I20261118 02:55:02.737749  4212 raft_consensus.cc:360] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.737929  4212 raft_consensus.cc:386] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.738003  4212 raft_consensus.cc:748] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.738220  4212 consensus_queue.cc:261] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.738754  4212 ts_tablet_manager.cc:1480] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.001s
I20261118 02:55:02.739086  4212 tablet_bootstrap.cc:492] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:02.739395  4210 raft_consensus.cc:521] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.740000  4210 leader_election.cc:290] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:02.740880  4212 tablet_bootstrap.cc:654] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.740824  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "65bd6fcaed5b47c3a2478d375ace9636" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:02.741031  3618 raft_consensus.cc:3096] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.742165  4213 tablet_bootstrap.cc:492] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:02.742398  4213 ts_tablet_manager.cc:1449] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.020s	user 0.001s	sys 0.003s
I20261118 02:55:02.743188  4213 raft_consensus.cc:360] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.743439  4213 raft_consensus.cc:386] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.743547  4213 raft_consensus.cc:748] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.743845  3618 raft_consensus.cc:2490] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:02.743884  4213 consensus_queue.cc:261] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.744406  4213 ts_tablet_manager.cc:1480] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20261118 02:55:02.745106  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "65bd6fcaed5b47c3a2478d375ace9636" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:02.745293  3755 raft_consensus.cc:2490] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:02.735004  4211 raft_consensus.cc:748] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.745702  4211 consensus_queue.cc:261] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.746233  4211 ts_tablet_manager.cc:1480] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.027s	user 0.001s	sys 0.000s
I20261118 02:55:02.746417  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "65bd6fcaed5b47c3a2478d375ace9636" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:02.746644  3752 raft_consensus.cc:3096] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.749362  4212 tablet_bootstrap.cc:492] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:02.749509  4212 ts_tablet_manager.cc:1449] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.010s	user 0.001s	sys 0.006s
I20261118 02:55:02.749681  3752 raft_consensus.cc:2490] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:02.750262  4212 raft_consensus.cc:360] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.750437  4212 raft_consensus.cc:386] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.750499  4212 raft_consensus.cc:748] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.750845  4211 tablet_bootstrap.cc:492] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:02.750751  4212 consensus_queue.cc:261] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.752421  4212 ts_tablet_manager.cc:1480] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:02.753146  3838 leader_election.cc:304] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:02.753475  4210 raft_consensus.cc:2826] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:02.753621  4210 raft_consensus.cc:705] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:02.753885  4210 consensus_queue.cc:238] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.761008  4211 tablet_bootstrap.cc:654] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.762140  4215 consensus_queue.cc:1049] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:02.762360  4210 consensus_queue.cc:1049] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:02.773140  4211 tablet_bootstrap.cc:492] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:02.773331  4211 ts_tablet_manager.cc:1449] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.023s	user 0.003s	sys 0.000s
I20261118 02:55:02.774183  4211 raft_consensus.cc:360] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:02.774386  4211 raft_consensus.cc:386] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.774549  4211 raft_consensus.cc:748] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.774864  4211 consensus_queue.cc:261] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:02.775645  4211 ts_tablet_manager.cc:1480] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.001s
I20261118 02:55:02.775959  4211 tablet_bootstrap.cc:492] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:02.777774  4211 tablet_bootstrap.cc:654] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.795586  4211 tablet_bootstrap.cc:492] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:02.796051  4211 ts_tablet_manager.cc:1449] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.004s
I20261118 02:55:02.796945  4211 raft_consensus.cc:360] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.797155  4211 raft_consensus.cc:386] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.797224  4211 raft_consensus.cc:748] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.797529  4211 consensus_queue.cc:261] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.798207  4211 ts_tablet_manager.cc:1480] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:02.798553  4211 tablet_bootstrap.cc:492] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:02.800755  4211 tablet_bootstrap.cc:654] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.804750  4211 tablet_bootstrap.cc:492] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:02.804915  4211 ts_tablet_manager.cc:1449] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.006s	user 0.003s	sys 0.000s
I20261118 02:55:02.805819  4211 raft_consensus.cc:360] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.806030  4211 raft_consensus.cc:386] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.806226  4211 raft_consensus.cc:748] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.806522  4211 consensus_queue.cc:261] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.807209  4211 ts_tablet_manager.cc:1480] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:02.807523  4211 tablet_bootstrap.cc:492] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:02.809410  4211 tablet_bootstrap.cc:654] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:02.839342  4211 tablet_bootstrap.cc:492] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:02.839573  4211 ts_tablet_manager.cc:1449] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.032s	user 0.003s	sys 0.000s
I20261118 02:55:02.840452  4211 raft_consensus.cc:360] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.840667  4211 raft_consensus.cc:386] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:02.840734  4211 raft_consensus.cc:748] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:02.841044  4211 consensus_queue.cc:261] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.841711  4211 ts_tablet_manager.cc:1480] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.003s	sys 0.000s
I20261118 02:55:02.893460  4215 raft_consensus.cc:499] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:02.893675  4215 raft_consensus.cc:521] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.894475  4215 leader_election.cc:290] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:02.895414  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "78189cccd457466bb5d01231038fec24" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:02.895464  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "78189cccd457466bb5d01231038fec24" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:02.895656  3618 raft_consensus.cc:2490] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:02.895713  3752 raft_consensus.cc:2490] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:02.896189  3838 leader_election.cc:304] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:02.896587  4215 raft_consensus.cc:2826] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:02.896709  4215 raft_consensus.cc:499] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:02.896813  4215 raft_consensus.cc:3096] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.899765  4215 raft_consensus.cc:521] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.900928  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "78189cccd457466bb5d01231038fec24" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:02.901141  3618 raft_consensus.cc:3096] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.904482  3618 raft_consensus.cc:2490] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:02.904999  3838 leader_election.cc:304] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:02.905334  4210 raft_consensus.cc:2826] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:02.905485  4210 raft_consensus.cc:705] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:02.906251  4210 consensus_queue.cc:238] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.908751  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "78189cccd457466bb5d01231038fec24" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:02.908952  3752 raft_consensus.cc:3096] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.911922  3752 raft_consensus.cc:2490] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:02.912911  4215 leader_election.cc:290] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:02.922106  4034 raft_consensus.cc:499] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:02.922333  4034 raft_consensus.cc:521] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.923112  4034 leader_election.cc:290] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:02.924456  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e71c5cd71ebf4fa6bd158d59ed2b191d" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:02.924696  3902 raft_consensus.cc:2490] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:02.925324  3553 leader_election.cc:304] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:02.925747  4034 raft_consensus.cc:2826] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:02.925832  4034 raft_consensus.cc:499] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:02.925917  4034 raft_consensus.cc:3096] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.928529  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e71c5cd71ebf4fa6bd158d59ed2b191d" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:02.928798  3752 raft_consensus.cc:2490] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:02.930457  4034 raft_consensus.cc:521] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.931615  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e71c5cd71ebf4fa6bd158d59ed2b191d" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:02.931833  3752 raft_consensus.cc:3096] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.934810  3752 raft_consensus.cc:2490] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:02.935365  3552 leader_election.cc:304] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:02.937553  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e71c5cd71ebf4fa6bd158d59ed2b191d" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:02.937777  3902 raft_consensus.cc:3096] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:02.941097  3902 raft_consensus.cc:2490] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:02.942000  4034 leader_election.cc:290] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:02.942567  4034 raft_consensus.cc:2826] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:02.942922  4034 raft_consensus.cc:705] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:02.946439  4034 consensus_queue.cc:238] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:02.955217  4215 consensus_queue.cc:1049] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:02.955566  4239 consensus_queue.cc:1049] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:02.961678  4263 consensus_queue.cc:1049] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.024931  4269 consensus_queue.cc:1049] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.079902  4207 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:03.080039  4207 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: loading cluster ID for follower catalog manager: success
I20261118 02:55:03.084290  4207 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: acquiring CA information for follower catalog manager: success
I20261118 02:55:03.087280  4207 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:55:03.121412  4263 raft_consensus.cc:499] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:03.121657  4263 raft_consensus.cc:521] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.122483  4263 leader_election.cc:290] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:03.123921  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e9323addc63245bfacb3854782425347" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:03.124187  3752 raft_consensus.cc:2490] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:03.124627  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e9323addc63245bfacb3854782425347" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:03.124868  3902 raft_consensus.cc:2490] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:03.125492  3553 leader_election.cc:304] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:03.126190  4263 raft_consensus.cc:2826] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:03.126302  4263 raft_consensus.cc:499] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:03.126416  4263 raft_consensus.cc:3096] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.130641  4263 raft_consensus.cc:521] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.132220  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e9323addc63245bfacb3854782425347" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:03.132436  3752 raft_consensus.cc:3096] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.135322  3752 raft_consensus.cc:2490] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:03.135979  3552 leader_election.cc:304] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:03.136355  4270 raft_consensus.cc:2826] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:03.136504  4270 raft_consensus.cc:705] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:03.136807  4270 consensus_queue.cc:238] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.150036  4274 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:55:03.150341  4274 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:03.151278  3902 raft_consensus.cc:3096] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.151767  4274 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:03.151852  4275 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:55:03.152185  4275 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:03.152758  3287 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 4 candidate_status { last_received { term: 3 index: 55 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:55:03.155781  3287 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 55 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:55:03.157490  4034 consensus_queue.cc:1049] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.159027  4034 consensus_queue.cc:1049] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.165925  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "e9323addc63245bfacb3854782425347" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:03.166877  4263 leader_election.cc:290] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:03.172950  4275 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
I20261118 02:55:03.174739  4161 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 4 candidate_status { last_received { term: 3 index: 55 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4" is_pre_election: true
I20261118 02:55:03.175145  4161 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 457eb38cba564ecd9f6f11245172260b in term 3.
I20261118 02:55:03.176805  3333 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d
I20261118 02:55:03.177156  4274 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Leader pre-election won for term 4
I20261118 02:55:03.177296  4274 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Starting leader election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:55:03.177381  4274 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 3 FOLLOWER]: Advancing to term 4
I20261118 02:55:03.180444  4274 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:03.181114  4274 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 4 election: Requested vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:03.182090  3287 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 4 candidate_status { last_received { term: 3 index: 55 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d"
I20261118 02:55:03.182473  4161 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 4 candidate_status { last_received { term: 3 index: 55 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4"
I20261118 02:55:03.182662  4161 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 3 FOLLOWER]: Advancing to term 4
I20261118 02:55:03.191579  4161 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 457eb38cba564ecd9f6f11245172260b in term 4.
I20261118 02:55:03.194694  3333 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d
I20261118 02:55:03.195053  4274 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 4 FOLLOWER]: Leader election won for term 4
I20261118 02:55:03.203677  3355 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 55 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b" is_pre_election: true
I20261118 02:55:03.204798  4161 raft_consensus.cc:1250] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Rejecting Update request from peer 26c6b20fa203404a88463ea27bdd3c7d for earlier term 3. Current term is 4. Ops: []
I20261118 02:55:03.213188  4283 consensus_queue.cc:1060] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 }, Status: INVALID_TERM, Last received: 3.55, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20261118 02:55:03.218750  4284 raft_consensus.cc:3091] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 3 LEADER]: Stepping down as leader of term 3
I20261118 02:55:03.219136  4284 raft_consensus.cc:748] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Running, Role: LEADER
I20261118 02:55:03.219581  4284 consensus_queue.cc:261] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 55, Committed index: 55, Last appended: 3.55, Last appended by leader: 55, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:03.220163  4284 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 3 FOLLOWER]: Advancing to term 4
I20261118 02:55:03.221344  4274 raft_consensus.cc:705] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 4 LEADER]: Becoming Leader. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Running, Role: LEADER
I20261118 02:55:03.221750  4274 consensus_queue.cc:238] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 55, Committed index: 55, Last appended: 3.55, Last appended by leader: 55, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:03.223515  4210 raft_consensus.cc:499] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:03.223692  4210 raft_consensus.cc:521] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.224463  4210 leader_election.cc:290] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:03.224998  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2de409a48f5b488484d74008f186d166" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:03.225231  3618 raft_consensus.cc:2490] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:03.225222  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2de409a48f5b488484d74008f186d166" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:03.225425  3755 raft_consensus.cc:2490] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:03.225742  3838 leader_election.cc:304] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:03.226023  4210 raft_consensus.cc:2826] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:03.226126  4210 raft_consensus.cc:499] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:03.226195  4210 raft_consensus.cc:3096] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.227938  4138 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:03.228355  4282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 4 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:03.228598  4282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:03.229053  4275 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20261118 02:55:03.229151  4210 raft_consensus.cc:521] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.230233  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2de409a48f5b488484d74008f186d166" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:03.230419  3618 raft_consensus.cc:3096] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.231324  4285 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:55:03.233618  3618 raft_consensus.cc:2490] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:03.234088  3838 leader_election.cc:304] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:03.234434  4239 raft_consensus.cc:2826] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:03.234560  4239 raft_consensus.cc:705] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:03.234835  4239 consensus_queue.cc:238] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.237493  4210 leader_election.cc:290] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:03.237653  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2de409a48f5b488484d74008f186d166" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:03.237876  3755 raft_consensus.cc:3096] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.240803  3755 raft_consensus.cc:2490] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:03.242805  4161 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Refusing update from remote peer 457eb38cba564ecd9f6f11245172260b: Log matching property violated. Preceding OpId in replica: term: 3 index: 55. Preceding OpId from leader: term: 4 index: 56. (index mismatch)
I20261118 02:55:03.244297  4282 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Connected to new peer: Peer: permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20261118 02:55:03.247650  4285 catalog_manager.cc:694] Loaded metadata for table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]
I20261118 02:55:03.248421  4285 catalog_manager.cc:694] Loaded metadata for table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]
I20261118 02:55:03.248888  4285 catalog_manager.cc:694] Loaded metadata for table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]
I20261118 02:55:03.249897  4285 catalog_manager.cc:694] Loaded metadata for table default.table_295a3e7e0d1f4e9db7078f6e5a5b9da1 [id=44b390866720455b8d2714eae004383b]
I20261118 02:55:03.250344  4285 catalog_manager.cc:694] Loaded metadata for table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]
I20261118 02:55:03.250787  4285 catalog_manager.cc:694] Loaded metadata for table default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da]
I20261118 02:55:03.251181  4285 catalog_manager.cc:694] Loaded metadata for table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
I20261118 02:55:03.251591  4285 catalog_manager.cc:694] Loaded metadata for table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:03.254302  3287 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 4 FOLLOWER]: Refusing update from remote peer 457eb38cba564ecd9f6f11245172260b: Log matching property violated. Preceding OpId in replica: term: 3 index: 55. Preceding OpId from leader: term: 4 index: 56. (index mismatch)
I20261118 02:55:03.255494  4288 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 56, Last known committed idx: 55, Time since last communication: 0.000s
I20261118 02:55:03.257939  4210 consensus_queue.cc:1049] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.258652  4284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 4 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:03.258842  4284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:03.259121  4275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 4 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:03.259346  4275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:03.260438  4210 consensus_queue.cc:1049] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.283639  4285 catalog_manager.cc:694] Loaded metadata for table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:03.284159  4285 catalog_manager.cc:694] Loaded metadata for table default.table_df2a47bb353f4cdcb4c16399a99cb722 [id=8a5ea4b95bee4db7af9ba875250d564b]
I20261118 02:55:03.284605  4285 catalog_manager.cc:694] Loaded metadata for table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]
I20261118 02:55:03.285006  4285 catalog_manager.cc:694] Loaded metadata for table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7]
I20261118 02:55:03.285444  4285 catalog_manager.cc:694] Loaded metadata for table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]
I20261118 02:55:03.285876  4285 catalog_manager.cc:694] Loaded metadata for table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:03.295509  4282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:03.296013  4274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:03.296324  4274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:03.296545  4282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:03.308308  4285 tablet_loader.cc:96] loaded metadata for tablet 1d347c0ae53241708ec623562f4740d4 (table default.table_df2a47bb353f4cdcb4c16399a99cb722 [id=8a5ea4b95bee4db7af9ba875250d564b])
I20261118 02:55:03.310734  4275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:03.310953  4275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:03.311246  4285 tablet_loader.cc:96] loaded metadata for tablet 2de409a48f5b488484d74008f186d166 (table default.table_0006608f28004098bb49ed40308f98fe [id=a2ece7af36934cdf80ad8458024adec7])
I20261118 02:55:03.311656  4285 tablet_loader.cc:96] loaded metadata for tablet 489beea71ae74236addee59c153cfee7 (table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e])
I20261118 02:55:03.312568  4285 tablet_loader.cc:96] loaded metadata for tablet 55f81e7b62e54ec18c7c13a200edb68f (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db])
I20261118 02:55:03.315150  4285 tablet_loader.cc:96] loaded metadata for tablet 5f05a8e5b21945a5bd3df86a5467a615 (table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf])
I20261118 02:55:03.315614  4285 tablet_loader.cc:96] loaded metadata for tablet 65bd6fcaed5b47c3a2478d375ace9636 (table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73])
I20261118 02:55:03.315896  4284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:03.316133  4284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:03.316834  4285 tablet_loader.cc:96] loaded metadata for tablet 78189cccd457466bb5d01231038fec24 (table default.table_f777ce1b15a4431fa208932de26f41fb [id=4a09f210503b499f8e842f6daafb33da])
I20261118 02:55:03.322645  4285 tablet_loader.cc:96] loaded metadata for tablet 8f853e6b923444efa79cfbab2e31540e (table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec])
I20261118 02:55:03.323632  4285 tablet_loader.cc:96] loaded metadata for tablet a013d180f2cd430ab1ba30b39fd27591 (table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d])
I20261118 02:55:03.324040  4285 tablet_loader.cc:96] loaded metadata for tablet a95a3f25901e468ab6e6cd02c2cd035f (table default.table_295a3e7e0d1f4e9db7078f6e5a5b9da1 [id=44b390866720455b8d2714eae004383b])
I20261118 02:55:03.324433  4285 tablet_loader.cc:96] loaded metadata for tablet c92c29c30ab6411dabe6f02331d29b7a (table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d])
I20261118 02:55:03.325109  4285 tablet_loader.cc:96] loaded metadata for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6])
I20261118 02:55:03.325471  4285 tablet_loader.cc:96] loaded metadata for tablet e9323addc63245bfacb3854782425347 (table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e])
I20261118 02:55:03.325857  4285 tablet_loader.cc:96] loaded metadata for tablet ffe8d4c111d14054b76f9c7724072a94 (table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66])
I20261118 02:55:03.326162  4285 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:55:03.327667  4285 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:03.327769  4285 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:55:03.329511  4285 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:55:03.341220  4239 raft_consensus.cc:499] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:03.341445  4239 raft_consensus.cc:521] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.342236  4239 leader_election.cc:290] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:03.343024  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "489beea71ae74236addee59c153cfee7" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:03.343281  3617 raft_consensus.cc:2490] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:03.343673  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "489beea71ae74236addee59c153cfee7" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:03.343817  3838 leader_election.cc:304] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:03.343917  3755 raft_consensus.cc:2490] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:03.345257  4285 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Loaded TSK: 0
I20261118 02:55:03.345374  4239 raft_consensus.cc:2826] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:03.345466  4239 raft_consensus.cc:499] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:03.345558  4239 raft_consensus.cc:3096] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.345836  4285 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:55:03.347131  4285 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:55:03.348455  4285 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 11
I20261118 02:55:03.348922  4239 raft_consensus.cc:521] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.350570  3617 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "489beea71ae74236addee59c153cfee7" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:03.350839  3617 raft_consensus.cc:3096] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.351061  3755 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "489beea71ae74236addee59c153cfee7" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:03.351289  3755 raft_consensus.cc:3096] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.352309  4239 leader_election.cc:290] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:03.354066  3755 raft_consensus.cc:2490] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:03.354327  3617 raft_consensus.cc:2490] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:03.354602  3836 leader_election.cc:304] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:03.355382  4239 raft_consensus.cc:2826] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:03.356189  4239 raft_consensus.cc:705] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:03.356560  4239 consensus_queue.cc:238] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.377735  3346 catalog_manager.cc:5840] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: HEALTHY } } }
I20261118 02:55:03.391562  4210 consensus_queue.cc:1049] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.399912  4239 consensus_queue.cc:1049] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 02:55:03.409021  3482 hms_client.cc:355] Time spent drop HMS table: real 1.047s	user 0.000s	sys 0.000s
I20261118 02:55:03.409365  3269 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]
I20261118 02:55:03.429870  4214 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:03.430089  4214 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:03.430902  4214 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:03.431723  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a013d180f2cd430ab1ba30b39fd27591" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:03.431933  3618 raft_consensus.cc:2490] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:03.432497  3691 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
W20261118 02:55:03.432767  3486 catalog_manager.cc:3980] Illegal state: an error occurred while updating the sys-catalog: replica 26c6b20fa203404a88463ea27bdd3c7d is not leader of this config: current role FOLLOWER
I20261118 02:55:03.432950  4214 raft_consensus.cc:2826] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:03.433051  4214 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20261118 02:55:03.433043  3486 hms_notification_log_listener.cc:330] Failed to handle Hive Metastore notification: 12 ALTER_TABLE default.table_30f082fbe850451b80d98df75ed1081a: Illegal state: an error occurred while updating the sys-catalog: replica 26c6b20fa203404a88463ea27bdd3c7d is not leader of this config: current role FOLLOWER
I20261118 02:55:03.433125  4214 raft_consensus.cc:3096] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.433245  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a013d180f2cd430ab1ba30b39fd27591" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:03.433480  3901 raft_consensus.cc:2490] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:03.436354  4214 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:03.437338  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a013d180f2cd430ab1ba30b39fd27591" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:03.437531  3901 raft_consensus.cc:3096] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.437681  3618 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a013d180f2cd430ab1ba30b39fd27591" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:03.437853  3618 raft_consensus.cc:3096] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.440456  3901 raft_consensus.cc:2490] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:03.440697  3618 raft_consensus.cc:2490] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:03.441155  3691 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:03.442617  4214 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:03.442819  4214 raft_consensus.cc:2826] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:03.442945  4214 raft_consensus.cc:705] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:03.443228  4214 consensus_queue.cc:238] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
W20261118 02:55:03.446483  3486 hms_notification_log_listener.cc:143] Hive Metastore notification log listener poll failed: Service unavailable: lost leadership while handling Hive Metastore notification log events: an error occurred while updating the sys-catalog: replica 26c6b20fa203404a88463ea27bdd3c7d is not leader of this config: current role FOLLOWER
I20261118 02:55:03.446903  3277 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 26) took 2767 ms. Trace:
I20261118 02:55:03.447062  3277 rpcz_store.cc:276] 1118 02:55:00.679452 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:01.871270 (+1191818us) service_pool.cc:224] Handling call
1118 02:55:03.446837 (+1575567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 02:55:03.447537  3275 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:52040 (request call id 23) took 2967 ms. Trace:
W20261118 02:55:03.447603  3270 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_51d36ff687ca47b5995e5693d7b8931b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 02:55:03.448141  3275 rpcz_store.cc:276] 1118 02:55:00.479705 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.888048 (+408343us) service_pool.cc:224] Handling call
1118 02:55:01.677032 (+788984us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.447026 (+1769994us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.447241 (+   215us) catalog_manager.cc:3766] Apply alter schema
1118 02:55:03.447490 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":9455,"Hive Metastore.run_cpu_time_us":655,"Hive Metastore.run_wall_time_us":194702,"spinlock_wait_cycles":2816}
W20261118 02:55:03.448364  3277 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.229.66:52737 (request call id 17) took 900 ms (client timeout 1000 ms). Trace:
W20261118 02:55:03.448493  3277 rpcz_store.cc:269] 1118 02:55:02.547455 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.447366 (+899911us) service_pool.cc:224] Handling call
1118 02:55:03.448321 (+   955us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 02:55:03.449167  3275 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.229.65:59011 (request call id 21) took 900 ms (client timeout 1000 ms). Trace:
W20261118 02:55:03.449273  3275 rpcz_store.cc:269] 1118 02:55:02.548572 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.448446 (+899874us) service_pool.cc:224] Handling call
1118 02:55:03.449125 (+   679us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 02:55:03.449621  3277 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.229.67:38945 (request call id 17) took 801 ms (client timeout 1000 ms). Trace:
I20261118 02:55:03.449750  3275 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 36) took 2514 ms. Trace:
I20261118 02:55:03.449872  3275 rpcz_store.cc:276] 1118 02:55:00.934962 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.449418 (+2514456us) service_pool.cc:224] Handling call
1118 02:55:03.449698 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 02:55:03.451026  3956 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.229.126:38025)
I20261118 02:55:03.451393  3274 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:52040 (request call id 12) took 3235 ms. Trace:
I20261118 02:55:03.451489  3274 rpcz_store.cc:276] 1118 02:55:00.216389 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.216823 (+   434us) service_pool.cc:224] Handling call
1118 02:55:01.677233 (+1460410us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.450814 (+1773581us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.451046 (+   232us) catalog_manager.cc:3766] Apply alter schema
1118 02:55:03.451357 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":204066,"Hive Metastore.run_cpu_time_us":790,"Hive Metastore.run_wall_time_us":83922}
I20261118 02:55:03.452023  3274 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 38) took 2383 ms. Trace:
I20261118 02:55:03.454147  3274 rpcz_store.cc:276] 1118 02:55:01.068231 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.451732 (+2383501us) service_pool.cc:224] Handling call
1118 02:55:03.451993 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20261118 02:55:03.449752  3277 rpcz_store.cc:269] 1118 02:55:02.647718 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.448750 (+801032us) service_pool.cc:224] Handling call
1118 02:55:03.449586 (+   836us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 02:55:03.455120  3272 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:52040 (request call id 17) took 3100 ms. Trace:
I20261118 02:55:03.455209  3272 rpcz_store.cc:276] 1118 02:55:00.352343 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.568189 (+215846us) service_pool.cc:224] Handling call
1118 02:55:01.677804 (+1109615us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.452204 (+1774400us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.452416 (+   212us) catalog_manager.cc:3766] Apply alter schema
1118 02:55:03.452656 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":287494,"Hive Metastore.run_cpu_time_us":777,"Hive Metastore.run_wall_time_us":79853}
I20261118 02:55:03.455560  3269 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:52040 (request call id 25) took 2816 ms. Trace:
I20261118 02:55:03.455972  3269 rpcz_store.cc:276] 1118 02:55:00.637027 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:01.076161 (+439134us) service_pool.cc:224] Handling call
1118 02:55:01.751224 (+675063us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.453248 (+1702024us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":610526,"Hive Metastore.run_cpu_time_us":387,"Hive Metastore.run_wall_time_us":1047193}
I20261118 02:55:03.456377  3344 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_f0ae490a0df64e1097e144c7a370df61"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:03.457382  3347 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_571d1360811c42dc815efb439b2b52e6"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:03.460268  3275 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:52040 (request call id 37) took 2433 ms. Trace:
I20261118 02:55:03.460486  3276 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:52040 (request call id 13) took 3211 ms. Trace:
I20261118 02:55:03.460832  3276 rpcz_store.cc:276] 1118 02:55:00.244156 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:00.244212 (+    56us) service_pool.cc:224] Handling call
1118 02:55:01.677032 (+1432820us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.452890 (+1775858us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:03.453077 (+   187us) catalog_manager.cc:3766] Apply alter schema
1118 02:55:03.455873 (+  2796us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":420452,"Hive Metastore.run_cpu_time_us":727,"Hive Metastore.run_wall_time_us":140665,"mutex_wait_us":1539}
I20261118 02:55:03.463210  3272 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:52040 (request call id 41) took 1505 ms. Trace:
I20261118 02:55:03.463317  3272 rpcz_store.cc:276] 1118 02:55:01.957099 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.455422 (+1498323us) service_pool.cc:224] Handling call
1118 02:55:03.461787 (+  6365us) spinlock_profiling.cc:245] Waited 5.75 ms on lock 0x616000043354. stack: 00007f9c4434a3a6 00007f9c4434a160 00007f9c4333b34d 00007f9c5a90e704 00007f9c5a90e6cd 00007f9c5386082d 00007f9c53860923 00007f9c503ca221 00007f9c582bf50d 00007f9c58450b06 00007f9c4ed5aea6 00007f9c4ed5abd5 00007f9c4b4928de 00007f9c4b4917be 00007f9c4b4979f8 00007f9c4b49835f
1118 02:55:03.462019 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13217152}
I20261118 02:55:03.463842  4300 consensus_queue.cc:1049] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.464421  4214 consensus_queue.cc:1049] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.465302  3272 master_service.cc:965] ReplaceTablet: received request to replace tablet a95a3f25901e468ab6e6cd02c2cd035f from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:03.466070  3272 master_service.cc:965] ReplaceTablet: received request to replace tablet a95a3f25901e468ab6e6cd02c2cd035f from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:03.466806  3272 master_service.cc:965] ReplaceTablet: received request to replace tablet 55f81e7b62e54ec18c7c13a200edb68f from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:03.467231  3272 master_service.cc:965] ReplaceTablet: received request to replace tablet 2de409a48f5b488484d74008f186d166 from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:03.466634  3275 rpcz_store.cc:276] 1118 02:55:01.027202 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.450042 (+2422840us) service_pool.cc:224] Handling call
1118 02:55:03.460228 (+ 10186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 02:55:03.468441  3269 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 42) took 1155 ms. Trace:
I20261118 02:55:03.468607  3272 master_service.cc:965] ReplaceTablet: received request to replace tablet e9323addc63245bfacb3854782425347 from {username='slave'} at 127.0.0.1:52040
I20261118 02:55:03.468904  3274 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 39) took 2355 ms. Trace:
I20261118 02:55:03.469182  3269 rpcz_store.cc:276] 1118 02:55:02.309431 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.456276 (+1146845us) service_pool.cc:224] Handling call
1118 02:55:03.462319 (+  6043us) spinlock_profiling.cc:245] Waited 5.81 ms on lock 0x616000043354. stack: 00007f9c4434a3a6 00007f9c4434a160 00007f9c4333b34d 00007f9c5a90e704 00007f9c5a90e6cd 00007f9c5386082d 00007f9c53860923 00007f9c503ca221 00007f9c582bf50d 00007f9c5844e873 00007f9c4ed57526 00007f9c4ed57255 00007f9c4b4928de 00007f9c4b4917be 00007f9c4b4979f8 00007f9c4b49835f
1118 02:55:03.465095 (+  2776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13373184}
I20261118 02:55:03.469892  3277 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 40) took 2237 ms. Trace:
I20261118 02:55:03.471904  3274 rpcz_store.cc:276] 1118 02:55:01.107611 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.454331 (+2346720us) service_pool.cc:224] Handling call
1118 02:55:03.462681 (+  8350us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 02:55:03.474150  3277 rpcz_store.cc:276] 1118 02:55:01.227610 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.454741 (+2227131us) service_pool.cc:224] Handling call
1118 02:55:03.462860 (+  8119us) spinlock_profiling.cc:245] Waited 7.69 ms on lock 0x616000043354. stack: 00007f9c4434a3a6 00007f9c4434a160 00007f9c4333b34d 00007f9c5a90e704 00007f9c5a90e6cd 00007f9c5386082d 00007f9c53860923 00007f9c503ca221 00007f9c582bf50d 00007f9c5844e873 00007f9c4ed57526 00007f9c4ed57255 00007f9c4b4928de 00007f9c4b4917be 00007f9c4b4979f8 00007f9c4b49835f
1118 02:55:03.465237 (+  2377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17687808}
I20261118 02:55:03.476303  3275 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:52050 (request call id 0) took 1338 ms. Trace:
I20261118 02:55:03.476471  3275 rpcz_store.cc:276] 1118 02:55:02.138073 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.471169 (+1333096us) service_pool.cc:224] Handling call
1118 02:55:03.476264 (+  5095us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20261118 02:55:03.481215  3949 heartbeater.cc:500] Master 127.2.229.125:38339 was elected leader, sending a full tablet report...
I20261118 02:55:03.482980  3802 heartbeater.cc:500] Master 127.2.229.125:38339 was elected leader, sending a full tablet report...
I20261118 02:55:03.487481  3345 catalog_manager.cc:5840] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66). New cstate: current_term: 1 leader_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: UNKNOWN } } }
I20261118 02:55:03.492015  3345 catalog_manager.cc:5840] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } }
I20261118 02:55:03.494798  3345 catalog_manager.cc:5840] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } }
I20261118 02:55:03.495751  3345 catalog_manager.cc:5940] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377) does not have the latest schema for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]). Expected version 1 got 0
I20261118 02:55:03.496626  3345 catalog_manager.cc:5840] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } }
I20261118 02:55:03.497951  3345 catalog_manager.cc:5840] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } }
I20261118 02:55:03.498889  3345 catalog_manager.cc:5840] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } }
I20261118 02:55:03.499925  3345 catalog_manager.cc:5840] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } }
I20261118 02:55:03.513034  3665 heartbeater.cc:500] Master 127.2.229.125:38339 was elected leader, sending a full tablet report...
I20261118 02:55:03.517779  3346 catalog_manager.cc:5940] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237) does not have the latest schema for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]). Expected version 1 got 0
I20261118 02:55:03.518090  3346 catalog_manager.cc:5940] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237) does not have the latest schema for tablet a95a3f25901e468ab6e6cd02c2cd035f (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]). Expected version 1 got 0
I20261118 02:55:03.521122  3343 catalog_manager.cc:5940] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705) does not have the latest schema for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]). Expected version 1 got 0
I20261118 02:55:03.521386  3343 catalog_manager.cc:5940] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705) does not have the latest schema for tablet a95a3f25901e468ab6e6cd02c2cd035f (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]). Expected version 1 got 0
I20261118 02:55:03.535491  3343 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_30b09a2798a54b01be7618b8a28eafaf"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20261118 02:55:03.545367  3956 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.2.229.125:38339)
I20261118 02:55:03.545638  3342 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_8ae376d4d4d64414b5c08dc3c2e58bc4" } new_table_name: "default.Table_5091d82298ca44bea5335d4989eb02d8" modify_external_catalogs: true
I20261118 02:55:03.552434  3341 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_61b6eaac749b4be0b0ddfba6ff9aa706"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:03.560573  3340 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_6e09d0e89581458183ed524cc1404849"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:03.564914  3339 master_service.cc:965] ReplaceTablet: received request to replace tablet a95a3f25901e468ab6e6cd02c2cd035f from {username='slave'} at 127.0.0.1:48092
W20261118 02:55:03.564642  3956 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.229.125:38339)
W20261118 02:55:03.579213  3691 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.229.65:33705, user_credentials={real_user=slave}} blocked reactor thread for 117228us
I20261118 02:55:03.579779  3345 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_38304cbb980440c2a8b8a6c13f49949c"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:03.582274  3339 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet a95a3f25901e468ab6e6cd02c2cd035f
I20261118 02:55:03.583256  3339 catalog_manager.cc:6936] ReplaceTablet: tablet a95a3f25901e468ab6e6cd02c2cd035f deleted and replaced by tablet 72f427c72a554ac1a0a2f2c3a78865da
W20261118 02:55:03.589473  3956 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.2.229.125:38339)
I20261118 02:55:03.591048  3339 master_service.cc:965] ReplaceTablet: received request to replace tablet a95a3f25901e468ab6e6cd02c2cd035f from {username='slave'} at 127.0.0.1:48092
I20261118 02:55:03.591394  3338 master_service.cc:965] ReplaceTablet: received request to replace tablet 2de409a48f5b488484d74008f186d166 from {username='slave'} at 127.0.0.1:48092
I20261118 02:55:03.592765  3339 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_28c62f411da74545aa42dad7325bc064" } modify_external_catalogs: true
I20261118 02:55:03.593070  3339 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_28c62f411da74545aa42dad7325bc064" } modify_external_catalogs: true
I20261118 02:55:03.593319  3346 master_service.cc:965] ReplaceTablet: received request to replace tablet 55f81e7b62e54ec18c7c13a200edb68f from {username='slave'} at 127.0.0.1:48092
I20261118 02:55:03.621941  3880 tablet_service.cc:1547] Processing DeleteTablet for tablet a95a3f25901e468ab6e6cd02c2cd035f with delete_type TABLET_DATA_DELETED (replaced by tablet 72f427c72a554ac1a0a2f2c3a78865da) from {username='slave'} at 127.0.0.1:50250
I20261118 02:55:03.622697  3879 tablet_service.cc:1500] Processing CreateTablet for tablet 72f427c72a554ac1a0a2f2c3a78865da (DEFAULT_TABLE table=default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.623270  3879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f427c72a554ac1a0a2f2c3a78865da. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.624219  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet a95a3f25901e468ab6e6cd02c2cd035f with delete_type TABLET_DATA_DELETED (replaced by tablet 72f427c72a554ac1a0a2f2c3a78865da) from {username='slave'} at 127.0.0.1:53002
I20261118 02:55:03.624902  3595 tablet_service.cc:1500] Processing CreateTablet for tablet 72f427c72a554ac1a0a2f2c3a78865da (DEFAULT_TABLE table=default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.625432  3595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f427c72a554ac1a0a2f2c3a78865da. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.666306  4321 tablet.cc:1759] T e71c5cd71ebf4fa6bd158d59ed2b191d P 1fdf7d7ece324fefb3a0c376add5b7e8: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.667302  4322 tablet.cc:1759] T e71c5cd71ebf4fa6bd158d59ed2b191d P 99a39b1d9d814deeb3a8a644e9b392ef: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.673839  3338 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet 2de409a48f5b488484d74008f186d166
I20261118 02:55:03.674403  3338 catalog_manager.cc:6936] ReplaceTablet: tablet 2de409a48f5b488484d74008f186d166 deleted and replaced by tablet 01778879aaa44ea9855fae67cfb3952d
I20261118 02:55:03.675971  3346 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet 55f81e7b62e54ec18c7c13a200edb68f
I20261118 02:55:03.676450  3346 catalog_manager.cc:6936] ReplaceTablet: tablet 55f81e7b62e54ec18c7c13a200edb68f deleted and replaced by tablet 2b01d7e596bd4790b8ba5dc88d8e12cc
I20261118 02:55:03.678709  3346 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_a14cb6414555446286bf35bc5c6fb910" } new_table_name: "default.Table_324a2600fef947748ef03e27f7cb7e09" modify_external_catalogs: true
I20261118 02:55:03.681584  3882 tablet_service.cc:1547] Processing DeleteTablet for tablet 2de409a48f5b488484d74008f186d166 with delete_type TABLET_DATA_DELETED (replaced by tablet 01778879aaa44ea9855fae67cfb3952d) from {username='slave'} at 127.0.0.1:50250
I20261118 02:55:03.682271  3882 tablet_service.cc:1547] Processing DeleteTablet for tablet 55f81e7b62e54ec18c7c13a200edb68f with delete_type TABLET_DATA_DELETED (replaced by tablet 2b01d7e596bd4790b8ba5dc88d8e12cc) from {username='slave'} at 127.0.0.1:50250
I20261118 02:55:03.683172  3338 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_d4fec0aaed15433ea373740872db53d9"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:03.683820  3527 catalog_manager.cc:2813] Deleting table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e] in response to Hive Metastore notification log event 20
I20261118 02:55:03.695082  3882 tablet_service.cc:1500] Processing CreateTablet for tablet 01778879aaa44ea9855fae67cfb3952d (DEFAULT_TABLE table=default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.695530  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01778879aaa44ea9855fae67cfb3952d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.704857  3527 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet e9323addc63245bfacb3854782425347
W20261118 02:55:03.706442  3344 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_f0ae490a0df64e1097e144c7a370df61 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:03.708050  3347 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_571d1360811c42dc815efb439b2b52e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20261118 02:55:03.708411  3878 tablet_service.cc:1547] Processing DeleteTablet for tablet e9323addc63245bfacb3854782425347 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 02:55:03 UTC) from {username='slave'} at 127.0.0.1:50250
I20261118 02:55:03.709728  3882 tablet_service.cc:1500] Processing CreateTablet for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (DEFAULT_TABLE table=default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.710244  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.731057  4321 tablet.cc:1759] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.745338  4333 tablet.cc:1759] T 78189cccd457466bb5d01231038fec24 P 3f1973986ec1466eab5bec01dd81654f: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.745716  3596 tablet_service.cc:1547] Processing DeleteTablet for tablet 2de409a48f5b488484d74008f186d166 with delete_type TABLET_DATA_DELETED (replaced by tablet 01778879aaa44ea9855fae67cfb3952d) from {username='slave'} at 127.0.0.1:53002
I20261118 02:55:03.755059  4336 tablet.cc:1759] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.759779  4337 tablet.cc:1759] T 78189cccd457466bb5d01231038fec24 P 99a39b1d9d814deeb3a8a644e9b392ef: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.761742  4322 tablet.cc:1759] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.765602  3593 tablet_service.cc:1547] Processing DeleteTablet for tablet 55f81e7b62e54ec18c7c13a200edb68f with delete_type TABLET_DATA_DELETED (replaced by tablet 2b01d7e596bd4790b8ba5dc88d8e12cc) from {username='slave'} at 127.0.0.1:53002
I20261118 02:55:03.766404  3593 tablet_service.cc:1500] Processing CreateTablet for tablet 01778879aaa44ea9855fae67cfb3952d (DEFAULT_TABLE table=default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.767023  3593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01778879aaa44ea9855fae67cfb3952d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.772742  3596 tablet_service.cc:1500] Processing CreateTablet for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (DEFAULT_TABLE table=default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.773170  3596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.774120  3592 tablet_service.cc:1547] Processing DeleteTablet for tablet e9323addc63245bfacb3854782425347 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 02:55:03 UTC) from {username='slave'} at 127.0.0.1:53002
I20261118 02:55:03.774791  3735 tablet_service.cc:1547] Processing DeleteTablet for tablet a95a3f25901e468ab6e6cd02c2cd035f with delete_type TABLET_DATA_DELETED (replaced by tablet 72f427c72a554ac1a0a2f2c3a78865da) from {username='slave'} at 127.0.0.1:57892
I20261118 02:55:03.779318  4338 tablet_replica.cc:333] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:03.779599  3735 tablet_service.cc:1500] Processing CreateTablet for tablet 72f427c72a554ac1a0a2f2c3a78865da (DEFAULT_TABLE table=default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.780071  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f427c72a554ac1a0a2f2c3a78865da. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.786352  4330 tablet.cc:1759] T 78189cccd457466bb5d01231038fec24 P 1fdf7d7ece324fefb3a0c376add5b7e8: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.792393  4330 tablet.cc:1759] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.799094  4333 tablet.cc:1759] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.799515  4335 tablet.cc:1759] T e71c5cd71ebf4fa6bd158d59ed2b191d P 3f1973986ec1466eab5bec01dd81654f: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.800868  4317 tablet_replica.cc:333] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:03.801453  4317 raft_consensus.cc:2261] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:03.801795  4317 raft_consensus.cc:2290] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.802723  4317 ts_tablet_manager.cc:1962] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.804030  4323 tablet_bootstrap.cc:492] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:03.804560  3734 tablet_service.cc:1547] Processing DeleteTablet for tablet 2de409a48f5b488484d74008f186d166 with delete_type TABLET_DATA_DELETED (replaced by tablet 01778879aaa44ea9855fae67cfb3952d) from {username='slave'} at 127.0.0.1:57892
I20261118 02:55:03.804627  3733 tablet_service.cc:1547] Processing DeleteTablet for tablet 55f81e7b62e54ec18c7c13a200edb68f with delete_type TABLET_DATA_DELETED (replaced by tablet 2b01d7e596bd4790b8ba5dc88d8e12cc) from {username='slave'} at 127.0.0.1:57892
I20261118 02:55:03.805130  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 01778879aaa44ea9855fae67cfb3952d (DEFAULT_TABLE table=default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.805495  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01778879aaa44ea9855fae67cfb3952d. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.806635  4323 tablet_bootstrap.cc:654] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.808066  4328 tablet_bootstrap.cc:492] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:03.810057  4328 tablet_bootstrap.cc:654] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.810580  4341 tablet_bootstrap.cc:492] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:03.811090  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (DEFAULT_TABLE table=default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:03.811463  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:03.814710  4341 tablet_bootstrap.cc:654] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.820399  4338 raft_consensus.cc:2261] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:03.820902  4338 raft_consensus.cc:2290] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.822979  4338 ts_tablet_manager.cc:1962] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.824615  4316 tablet_replica.cc:333] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:03.824888  4316 raft_consensus.cc:2261] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:03.825148  4316 pending_rounds.cc:70] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Trying to abort 1 pending ops.
I20261118 02:55:03.825354  4316 pending_rounds.cc:77] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7352199182850248704 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "a95a3f25901e468ab6e6cd02c2cd035f" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.table_30f082fbe850451b80d98df75ed1081a" dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" new_extra_config { } }
I20261118 02:55:03.825989  4316 raft_consensus.cc:2290] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.830727  4338 ts_tablet_manager.cc:1975] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:03.830931  4338 log.cc:1199] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/a95a3f25901e468ab6e6cd02c2cd035f
I20261118 02:55:03.831468  4338 ts_tablet_manager.cc:1996] T a95a3f25901e468ab6e6cd02c2cd035f P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:03.832007  4338 tablet_replica.cc:333] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:03.832305  4338 raft_consensus.cc:2261] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:03.832566  4338 raft_consensus.cc:2290] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.833137  3332 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet a95a3f25901e468ab6e6cd02c2cd035f (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]) successfully deleted
I20261118 02:55:03.834319  4338 ts_tablet_manager.cc:1962] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.835621  4321 tablet.cc:1759] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.836117  4316 ts_tablet_manager.cc:1962] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.840148  3735 tablet_service.cc:1547] Processing DeleteTablet for tablet e9323addc63245bfacb3854782425347 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-11-18 02:55:03 UTC) from {username='slave'} at 127.0.0.1:57892
I20261118 02:55:03.841574  4338 ts_tablet_manager.cc:1975] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:03.841698  4338 log.cc:1199] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/2de409a48f5b488484d74008f186d166
I20261118 02:55:03.842047  4338 ts_tablet_manager.cc:1996] T 2de409a48f5b488484d74008f186d166 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:03.842489  4338 tablet_replica.cc:333] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:03.842674  4338 raft_consensus.cc:2261] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:03.842864  4338 raft_consensus.cc:2290] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.843048  3332 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet 2de409a48f5b488484d74008f186d166 (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
I20261118 02:55:03.843485  4335 tablet.cc:1759] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.844619  4328 tablet_bootstrap.cc:492] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:03.844769  4328 ts_tablet_manager.cc:1449] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.037s	user 0.000s	sys 0.003s
I20261118 02:55:03.845530  4328 raft_consensus.cc:360] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.845713  4328 raft_consensus.cc:386] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:03.845803  4328 raft_consensus.cc:748] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:03.846498  4328 consensus_queue.cc:261] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.847155  4328 ts_tablet_manager.cc:1480] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:03.848531  4323 tablet_bootstrap.cc:492] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:03.848742  4323 ts_tablet_manager.cc:1449] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.045s	user 0.003s	sys 0.001s
I20261118 02:55:03.849772  4323 raft_consensus.cc:360] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.850015  4323 raft_consensus.cc:386] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:03.850136  4323 raft_consensus.cc:748] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:03.850394  4323 consensus_queue.cc:261] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.851122  4323 ts_tablet_manager.cc:1480] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.002s
I20261118 02:55:03.851469  4323 tablet_bootstrap.cc:492] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:03.853374  4323 tablet_bootstrap.cc:654] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.857371  4341 tablet_bootstrap.cc:492] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:03.857643  4341 ts_tablet_manager.cc:1449] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.047s	user 0.000s	sys 0.011s
I20261118 02:55:03.858532  4341 raft_consensus.cc:360] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.858798  4341 raft_consensus.cc:386] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:03.858952  4341 raft_consensus.cc:748] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:03.859251  4341 consensus_queue.cc:261] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.860420  4332 tablet.cc:1759] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:03.861956  4341 ts_tablet_manager.cc:1480] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.002s
I20261118 02:55:03.863229  4341 tablet_bootstrap.cc:492] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:03.867128  4341 tablet_bootstrap.cc:654] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.869217  4338 ts_tablet_manager.cc:1962] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.875099  4323 tablet_bootstrap.cc:492] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:03.875352  4323 ts_tablet_manager.cc:1449] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.001s
I20261118 02:55:03.876965  4323 raft_consensus.cc:360] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.877614  4323 raft_consensus.cc:386] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:03.877737  4323 raft_consensus.cc:748] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:03.880014  4323 consensus_queue.cc:261] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.880988  4323 ts_tablet_manager.cc:1480] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 02:55:03.881300  4323 tablet_bootstrap.cc:492] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:03.881603  4325 raft_consensus.cc:499] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:03.877444  4317 ts_tablet_manager.cc:1975] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:03.881888  4317 log.cc:1199] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/a95a3f25901e468ab6e6cd02c2cd035f
I20261118 02:55:03.881789  4325 raft_consensus.cc:521] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.882475  4317 ts_tablet_manager.cc:1996] T a95a3f25901e468ab6e6cd02c2cd035f P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:03.882546  4325 leader_election.cc:290] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:03.883306  4328 tablet_bootstrap.cc:492] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:03.884974  4323 tablet_bootstrap.cc:654] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.885033  4328 tablet_bootstrap.cc:654] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.887609  3753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "72f427c72a554ac1a0a2f2c3a78865da" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
W20261118 02:55:03.888275  3836 leader_election.cc:343] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Illegal state: must be running to vote when last-logged opid is not known
I20261118 02:55:03.889134  4338 ts_tablet_manager.cc:1975] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:03.889251  4338 log.cc:1199] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/55f81e7b62e54ec18c7c13a200edb68f
I20261118 02:55:03.889629  4338 ts_tablet_manager.cc:1996] T 55f81e7b62e54ec18c7c13a200edb68f P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:03.890619  3332 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet 55f81e7b62e54ec18c7c13a200edb68f (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]) successfully deleted
I20261118 02:55:03.891647  4341 tablet_bootstrap.cc:492] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:03.892009  4341 ts_tablet_manager.cc:1449] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.029s	user 0.000s	sys 0.005s
I20261118 02:55:03.892843  4341 raft_consensus.cc:360] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.893030  4341 raft_consensus.cc:386] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:03.893100  4341 raft_consensus.cc:748] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:03.893326  4341 consensus_queue.cc:261] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.893555  3334 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet a95a3f25901e468ab6e6cd02c2cd035f (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]) successfully deleted
I20261118 02:55:03.894984  4341 ts_tablet_manager.cc:1480] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:03.885650  4317 tablet_replica.cc:333] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:03.894260  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "72f427c72a554ac1a0a2f2c3a78865da" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:03.896133  3616 raft_consensus.cc:2490] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:03.897914  4317 raft_consensus.cc:2261] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:03.898124  4317 raft_consensus.cc:2290] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.898994  4317 ts_tablet_manager.cc:1962] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.899885  4341 tablet_bootstrap.cc:492] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:03.901618  4341 tablet_bootstrap.cc:654] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.902745  3838 leader_election.cc:304] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:03.903264  4325 raft_consensus.cc:2826] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:03.903398  4325 raft_consensus.cc:499] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:03.903484  4325 raft_consensus.cc:3096] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.907677  4338 tablet_replica.cc:333] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:03.907611  4325 raft_consensus.cc:521] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.907984  4338 raft_consensus.cc:2261] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:03.908247  4338 raft_consensus.cc:2290] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.909160  3753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "72f427c72a554ac1a0a2f2c3a78865da" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:03.909413  3753 raft_consensus.cc:3096] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.910567  4338 ts_tablet_manager.cc:1962] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.912290  4325 leader_election.cc:290] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:03.912967  3753 raft_consensus.cc:2490] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:03.913532  3836 leader_election.cc:304] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:03.917097  4325 raft_consensus.cc:2826] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:03.917665  4325 raft_consensus.cc:705] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:03.918107  4325 consensus_queue.cc:238] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:03.918912  4341 tablet_bootstrap.cc:492] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:03.919047  4341 ts_tablet_manager.cc:1449] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.000s
I20261118 02:55:03.919775  4341 raft_consensus.cc:360] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.919951  4341 raft_consensus.cc:386] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:03.920032  4341 raft_consensus.cc:748] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:03.920293  4341 consensus_queue.cc:261] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.920884  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "72f427c72a554ac1a0a2f2c3a78865da" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:03.921077  3616 raft_consensus.cc:3096] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:03.924714  3616 raft_consensus.cc:2490] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:03.925630  4317 ts_tablet_manager.cc:1975] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:03.925784  4317 log.cc:1199] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/2de409a48f5b488484d74008f186d166
I20261118 02:55:03.926209  4317 ts_tablet_manager.cc:1996] T 2de409a48f5b488484d74008f186d166 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:03.927208  4317 tablet_replica.cc:333] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:03.927430  4317 raft_consensus.cc:2261] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:03.927634  4317 raft_consensus.cc:2290] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.927621  3334 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet 2de409a48f5b488484d74008f186d166 (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
I20261118 02:55:03.927918  4341 ts_tablet_manager.cc:1480] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.000s
I20261118 02:55:03.934151  4338 ts_tablet_manager.cc:1975] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:03.934402  4338 log.cc:1199] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/e9323addc63245bfacb3854782425347
I20261118 02:55:03.934832  4338 ts_tablet_manager.cc:1996] T e9323addc63245bfacb3854782425347 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:03.936023  3332 catalog_manager.cc:5065] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): tablet e9323addc63245bfacb3854782425347 (table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]) successfully deleted
I20261118 02:55:03.937003  4323 tablet_bootstrap.cc:492] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:03.937216  4323 ts_tablet_manager.cc:1449] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.056s	user 0.007s	sys 0.003s
I20261118 02:55:03.938122  4323 raft_consensus.cc:360] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.938447  4323 raft_consensus.cc:386] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:03.938594  4323 raft_consensus.cc:748] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:03.938946  4323 consensus_queue.cc:261] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.939652  4215 consensus_queue.cc:1049] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.941550  4323 ts_tablet_manager.cc:1480] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.000s
I20261118 02:55:03.944454  4317 ts_tablet_manager.cc:1962] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.955156  4317 ts_tablet_manager.cc:1975] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:03.955300  4317 log.cc:1199] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/55f81e7b62e54ec18c7c13a200edb68f
I20261118 02:55:03.955766  4317 ts_tablet_manager.cc:1996] T 55f81e7b62e54ec18c7c13a200edb68f P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:03.956708  4317 tablet_replica.cc:333] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:03.956945  4317 raft_consensus.cc:2261] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:03.957273  4317 raft_consensus.cc:2290] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:03.957788  3334 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet 55f81e7b62e54ec18c7c13a200edb68f (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]) successfully deleted
I20261118 02:55:03.958659  4325 consensus_queue.cc:1049] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:03.966035  4328 tablet_bootstrap.cc:492] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:03.966246  4328 ts_tablet_manager.cc:1449] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.083s	user 0.018s	sys 0.014s
I20261118 02:55:03.969515  4328 raft_consensus.cc:360] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.969734  4328 raft_consensus.cc:386] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:03.969825  4328 raft_consensus.cc:748] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:03.970053  4328 consensus_queue.cc:261] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:03.970542  4328 ts_tablet_manager.cc:1480] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.001s
I20261118 02:55:03.971829  4317 ts_tablet_manager.cc:1962] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:03.974143  4328 tablet_bootstrap.cc:492] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:03.989940  4316 ts_tablet_manager.cc:1975] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:03.990080  4316 log.cc:1199] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/a95a3f25901e468ab6e6cd02c2cd035f
I20261118 02:55:03.991266  4328 tablet_bootstrap.cc:654] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:03.991525  4316 ts_tablet_manager.cc:1996] T a95a3f25901e468ab6e6cd02c2cd035f P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:03.992662  3333 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet a95a3f25901e468ab6e6cd02c2cd035f (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]) successfully deleted
I20261118 02:55:03.999418  4316 tablet_replica.cc:333] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:03.999660  4316 raft_consensus.cc:2261] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:04.000298  4316 raft_consensus.cc:2290] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:04.001791  4317 ts_tablet_manager.cc:1975] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:04.002020  4317 log.cc:1199] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/e9323addc63245bfacb3854782425347
I20261118 02:55:04.002501  4317 ts_tablet_manager.cc:1996] T e9323addc63245bfacb3854782425347 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:04.003826  3334 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet e9323addc63245bfacb3854782425347 (table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]) successfully deleted
I20261118 02:55:04.004763  4316 ts_tablet_manager.cc:1962] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:04.011252  4328 tablet_bootstrap.cc:492] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:04.011423  4328 ts_tablet_manager.cc:1449] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.037s	user 0.002s	sys 0.002s
I20261118 02:55:04.012310  4328 raft_consensus.cc:360] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.012499  4328 raft_consensus.cc:386] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.012580  4328 raft_consensus.cc:748] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.012864  4328 consensus_queue.cc:261] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.013844  4328 ts_tablet_manager.cc:1480] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:04.021203  3268 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:04.021359  4316 ts_tablet_manager.cc:1975] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:04.021528  4316 log.cc:1199] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/2de409a48f5b488484d74008f186d166
I20261118 02:55:04.021843  3268 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:52040 (request call id 33) took 3151 ms. Trace:
I20261118 02:55:04.021916  4316 ts_tablet_manager.cc:1996] T 2de409a48f5b488484d74008f186d166 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:04.021958  3268 rpcz_store.cc:276] 1118 02:55:00.869974 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:02.404597 (+1534623us) service_pool.cc:224] Handling call
1118 02:55:03.446961 (+1042364us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:04.021807 (+574846us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1569,"Hive Metastore.run_cpu_time_us":205,"Hive Metastore.run_wall_time_us":572184}
I20261118 02:55:04.022964  4316 tablet_replica.cc:333] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:04.023159  4316 raft_consensus.cc:2261] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:04.023161  3333 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet 2de409a48f5b488484d74008f186d166 (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]) successfully deleted
I20261118 02:55:04.023511  4316 raft_consensus.cc:2290] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:04.024121  4316 ts_tablet_manager.cc:1962] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:04.030874  4316 ts_tablet_manager.cc:1975] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:04.031024  4316 log.cc:1199] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/55f81e7b62e54ec18c7c13a200edb68f
I20261118 02:55:04.031443  4316 ts_tablet_manager.cc:1996] T 55f81e7b62e54ec18c7c13a200edb68f P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:04.032509  4316 tablet_replica.cc:333] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:04.033041  3333 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet 55f81e7b62e54ec18c7c13a200edb68f (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]) successfully deleted
I20261118 02:55:04.033195  4316 raft_consensus.cc:2261] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:04.033378  4316 raft_consensus.cc:2290] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:04.033366  3344 catalog_manager.cc:2215] created HMS catalog entry for table  [id=a59b9425c01049269883cf7767936c67]
I20261118 02:55:04.036355  4316 ts_tablet_manager.cc:1962] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:04.057052  4316 ts_tablet_manager.cc:1975] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:04.057224  4316 log.cc:1199] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/e9323addc63245bfacb3854782425347
I20261118 02:55:04.057595  4316 ts_tablet_manager.cc:1996] T e9323addc63245bfacb3854782425347 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:04.067538  3597 tablet_service.cc:1500] Processing CreateTablet for tablet f094af4ac3604326a539461d06b01868 (DEFAULT_TABLE table=default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.068518  3734 tablet_service.cc:1500] Processing CreateTablet for tablet f094af4ac3604326a539461d06b01868 (DEFAULT_TABLE table=default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.069027  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f094af4ac3604326a539461d06b01868. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.072523  3880 tablet_service.cc:1500] Processing CreateTablet for tablet f094af4ac3604326a539461d06b01868 (DEFAULT_TABLE table=default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.073032  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f094af4ac3604326a539461d06b01868. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.076151  4265 raft_consensus.cc:499] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:04.076321  4265 raft_consensus.cc:521] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.068042  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f094af4ac3604326a539461d06b01868. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.077124  4265 leader_election.cc:290] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.078401  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "01778879aaa44ea9855fae67cfb3952d" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:04.078657  3752 raft_consensus.cc:2490] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:04.079171  3552 leader_election.cc:304] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.079885  4265 raft_consensus.cc:2826] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:04.079988  4265 raft_consensus.cc:499] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:04.080070  4265 raft_consensus.cc:3096] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.079994  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "01778879aaa44ea9855fae67cfb3952d" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:04.080575  3902 raft_consensus.cc:2490] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:04.081190  4325 raft_consensus.cc:499] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:04.081338  4325 raft_consensus.cc:521] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.081983  4325 leader_election.cc:290] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:04.082883  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2b01d7e596bd4790b8ba5dc88d8e12cc" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:04.082916  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2b01d7e596bd4790b8ba5dc88d8e12cc" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:04.083098  3616 raft_consensus.cc:2490] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:04.083117  3752 raft_consensus.cc:2490] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:04.083560  3836 leader_election.cc:304] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.083521  4265 raft_consensus.cc:521] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.084262  4325 raft_consensus.cc:2826] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:04.084379  4325 raft_consensus.cc:499] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:04.084477  4325 raft_consensus.cc:3096] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.084460  3752 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "01778879aaa44ea9855fae67cfb3952d" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:04.084643  3752 raft_consensus.cc:3096] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.087608  3752 raft_consensus.cc:2490] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:04.087842  4325 raft_consensus.cc:521] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.088083  3552 leader_election.cc:304] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.088405  4270 raft_consensus.cc:2826] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:04.088578  4270 raft_consensus.cc:705] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:04.088869  4270 consensus_queue.cc:238] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.090488  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2b01d7e596bd4790b8ba5dc88d8e12cc" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:04.090677  3616 raft_consensus.cc:3096] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.093236  3616 raft_consensus.cc:2490] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:04.093619  4341 tablet_bootstrap.cc:492] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:04.094107  3838 leader_election.cc:304] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:04.094774  4215 raft_consensus.cc:2826] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:04.094908  4215 raft_consensus.cc:705] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:04.095204  4215 consensus_queue.cc:238] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.095428  4341 tablet_bootstrap.cc:654] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.096323  4328 tablet_bootstrap.cc:492] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:04.096511  4323 tablet_bootstrap.cc:492] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:04.098119  4328 tablet_bootstrap.cc:654] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.098690  4323 tablet_bootstrap.cc:654] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.108286  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "01778879aaa44ea9855fae67cfb3952d" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:04.108562  3902 raft_consensus.cc:3096] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.109030  3753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2b01d7e596bd4790b8ba5dc88d8e12cc" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:04.109234  3753 raft_consensus.cc:3096] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.111523  3902 raft_consensus.cc:2490] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:04.112910  3753 raft_consensus.cc:2490] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:04.113003  4265 leader_election.cc:290] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.114075  4325 leader_election.cc:290] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:04.118269  3344 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da] alter complete (version 1)
I20261118 02:55:04.120646  3344 catalog_manager.cc:5940] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705) does not have the latest schema for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]). Expected version 1 got 0
I20261118 02:55:04.121176  3344 catalog_manager.cc:5840] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } }
I20261118 02:55:04.122175  3344 catalog_manager.cc:5940] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705) does not have the latest schema for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]). Expected version 1 got 0
I20261118 02:55:04.128289  3333 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet e9323addc63245bfacb3854782425347 (table default.table_bca0059ea2004174b89866f3e8cc68fb [id=469b750094124f159d835e6fb5a6486e]) successfully deleted
I20261118 02:55:04.132853  4328 tablet_bootstrap.cc:492] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:04.133052  4328 ts_tablet_manager.cc:1449] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.037s	user 0.004s	sys 0.007s
I20261118 02:55:04.134003  4328 raft_consensus.cc:360] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.134217  4328 raft_consensus.cc:386] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.134294  4328 raft_consensus.cc:748] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.142890  4210 consensus_queue.cc:1049] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.151130  4215 consensus_queue.cc:1049] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.170456  4341 tablet_bootstrap.cc:492] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:04.170768  4341 ts_tablet_manager.cc:1449] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.003s
I20261118 02:55:04.173640  4341 raft_consensus.cc:360] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.173959  4341 raft_consensus.cc:386] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.175185  4323 tablet_bootstrap.cc:492] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:04.175429  4323 ts_tablet_manager.cc:1449] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.079s	user 0.005s	sys 0.006s
I20261118 02:55:04.176498  4323 raft_consensus.cc:360] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.176769  4323 raft_consensus.cc:386] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.176898  4323 raft_consensus.cc:748] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.177419  4323 consensus_queue.cc:261] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.183075  4270 consensus_queue.cc:1049] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20261118 02:55:04.183395  4265 consensus_queue.cc:1049] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.174098  4341 raft_consensus.cc:748] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.196213  4323 ts_tablet_manager.cc:1480] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.000s
I20261118 02:55:04.195526  4341 consensus_queue.cc:261] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.210675  4341 ts_tablet_manager.cc:1480] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.039s	user 0.002s	sys 0.000s
I20261118 02:55:04.221591  3344 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b] alter complete (version 1)
W20261118 02:55:04.226326  3344 catalog_manager.cc:6040] Failed to send AlterTable RPC for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 01778879aaa44ea9855fae67cfb3952d consensus state has no leader
I20261118 02:55:04.232226  4328 consensus_queue.cc:261] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.232650  4333 tablet.cc:1759] T 72f427c72a554ac1a0a2f2c3a78865da P 3f1973986ec1466eab5bec01dd81654f: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:04.232920  4328 ts_tablet_manager.cc:1480] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.100s	user 0.002s	sys 0.000s
I20261118 02:55:04.242697  4330 tablet.cc:1759] T 72f427c72a554ac1a0a2f2c3a78865da P 1fdf7d7ece324fefb3a0c376add5b7e8: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:04.250113  4332 tablet.cc:1759] T 72f427c72a554ac1a0a2f2c3a78865da P 99a39b1d9d814deeb3a8a644e9b392ef: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:04.250375  4325 raft_consensus.cc:499] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:04.253129  4325 raft_consensus.cc:521] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.254190  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f094af4ac3604326a539461d06b01868" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:04.254416  3616 raft_consensus.cc:2490] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:04.255401  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f094af4ac3604326a539461d06b01868" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:04.255657  3751 raft_consensus.cc:2490] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:04.256052  3838 leader_election.cc:304] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:04.256448  4210 raft_consensus.cc:2826] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:04.256536  4325 leader_election.cc:290] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:04.256567  4210 raft_consensus.cc:499] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:04.256664  4210 raft_consensus.cc:3096] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.259512  4210 raft_consensus.cc:521] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.260694  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f094af4ac3604326a539461d06b01868" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:04.260905  3616 raft_consensus.cc:3096] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.253939  3344 catalog_manager.cc:5940] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377) does not have the latest schema for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]). Expected version 1 got 0
I20261118 02:55:04.261639  4210 leader_election.cc:290] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:04.262578  3344 catalog_manager.cc:5940] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377) does not have the latest schema for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]). Expected version 1 got 0
I20261118 02:55:04.263835  3616 raft_consensus.cc:2490] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:04.264951  3838 leader_election.cc:304] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:04.265004  3347 catalog_manager.cc:2215] created HMS catalog entry for table  [id=e797fad7075b40d9bdd674a0e0023139]
I20261118 02:55:04.265265  4210 raft_consensus.cc:2826] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:04.265422  4210 raft_consensus.cc:705] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:04.265718  4210 consensus_queue.cc:238] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.267365  3270 catalog_manager.cc:2215] created HMS catalog entry for table  [id=baa54e879f6246509f1c5ca1876f0b25]
W20261118 02:55:04.268260  3270 catalog_manager.cc:2243] Illegal state: an error occurred while writing to the sys-catalog: replica 26c6b20fa203404a88463ea27bdd3c7d is not leader of this config: current role FOLLOWER
I20261118 02:55:04.269680  3344 catalog_manager.cc:5940] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237) does not have the latest schema for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]). Expected version 1 got 0
I20261118 02:55:04.270011  3344 catalog_manager.cc:5940] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237) does not have the latest schema for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]). Expected version 1 got 0
I20261118 02:55:04.270249  3344 catalog_manager.cc:5840] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: HEALTHY } } }
W20261118 02:55:04.275611  3341 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:04.275942  3343 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_30b09a2798a54b01be7618b8a28eafaf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:04.276232  3345 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_38304cbb980440c2a8b8a6c13f49949c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:04.277308  3340 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_6e09d0e89581458183ed524cc1404849 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:04.277547  3338 catalog_manager.cc:7228] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d4fec0aaed15433ea373740872db53d9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20261118 02:55:04.281633  3334 catalog_manager.cc:4847] Async tablet task AlterTable RPC for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 01778879aaa44ea9855fae67cfb3952d consensus state has no leader
I20261118 02:55:04.297396  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "f094af4ac3604326a539461d06b01868" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:04.298947  3751 raft_consensus.cc:3096] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.302254  3751 raft_consensus.cc:2490] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:04.312664  3880 tablet_service.cc:1500] Processing CreateTablet for tablet b610080fc91745c7be260699946d5681 (DEFAULT_TABLE table=default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.313160  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b610080fc91745c7be260699946d5681. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.318404  3347 catalog_manager.cc:5840] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: UNKNOWN } } }
I20261118 02:55:04.318907  3347 catalog_manager.cc:5940] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705) does not have the latest schema for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]). Expected version 1 got 0
I20261118 02:55:04.320631  3734 tablet_service.cc:1500] Processing CreateTablet for tablet b610080fc91745c7be260699946d5681 (DEFAULT_TABLE table=default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.321077  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b610080fc91745c7be260699946d5681. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.325342  3597 tablet_service.cc:1500] Processing CreateTablet for tablet b610080fc91745c7be260699946d5681 (DEFAULT_TABLE table=default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.325804  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b610080fc91745c7be260699946d5681. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.327931  4210 consensus_queue.cc:1049] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.328421  4210 consensus_queue.cc:1049] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.331703  3347 catalog_manager.cc:5940] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237) does not have the latest schema for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]). Expected version 1 got 0
I20261118 02:55:04.332062  3347 catalog_manager.cc:5840] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f1973986ec1466eab5bec01dd81654f (127.2.229.67). New cstate: current_term: 1 leader_uuid: "3f1973986ec1466eab5bec01dd81654f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: HEALTHY } } }
I20261118 02:55:04.334553  4323 tablet_bootstrap.cc:492] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:04.336999  4323 tablet_bootstrap.cc:654] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.338425  4328 tablet_bootstrap.cc:492] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:04.338806  4341 tablet_bootstrap.cc:492] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:04.340728  4341 tablet_bootstrap.cc:654] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.344504  4328 tablet_bootstrap.cc:654] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.344513  3344 catalog_manager.cc:5940] TS 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377) does not have the latest schema for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]). Expected version 1 got 0
I20261118 02:55:04.349254  4323 tablet_bootstrap.cc:492] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:04.349424  4323 ts_tablet_manager.cc:1449] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.015s	user 0.003s	sys 0.002s
I20261118 02:55:04.350948  3347 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b] alter complete (version 1)
I20261118 02:55:04.350237  4323 raft_consensus.cc:360] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.351199  4323 raft_consensus.cc:386] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.351294  4323 raft_consensus.cc:748] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.351647  3347 master_service.cc:965] ReplaceTablet: received request to replace tablet e9323addc63245bfacb3854782425347 from {username='slave'} at 127.0.0.1:48092
I20261118 02:55:04.351619  4323 consensus_queue.cc:261] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.352317  4323 ts_tablet_manager.cc:1480] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.001s
I20261118 02:55:04.352272  3347 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_e39f59e2fdbb47fda4c10beba29ad8f1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:04.358325  3344 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48092:
name: "default.Table_fd4fcac1f1d946fa898176f1d5e87e9b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:04.377179  4341 tablet_bootstrap.cc:492] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:04.377388  4341 ts_tablet_manager.cc:1449] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.000s
I20261118 02:55:04.378207  4341 raft_consensus.cc:360] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.378402  4341 raft_consensus.cc:386] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.378512  4341 raft_consensus.cc:748] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.379442  4328 tablet_bootstrap.cc:492] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:04.379642  4328 ts_tablet_manager.cc:1449] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.041s	user 0.001s	sys 0.003s
I20261118 02:55:04.380507  4328 raft_consensus.cc:360] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.380713  4328 raft_consensus.cc:386] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.380808  4328 raft_consensus.cc:748] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.381094  4328 consensus_queue.cc:261] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.385006  4328 ts_tablet_manager.cc:1480] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 02:55:04.406440  4341 consensus_queue.cc:261] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.413273  4341 ts_tablet_manager.cc:1480] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.036s	user 0.002s	sys 0.001s
I20261118 02:55:04.417950  3341 catalog_manager.cc:2215] created HMS catalog entry for table  [id=e0ed3867fcbb42a4b0133f38e0372a6f]
I20261118 02:55:04.425582  4330 tablet.cc:1759] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:04.430912  3341 catalog_manager.cc:5940] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705) does not have the latest schema for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]). Expected version 1 got 0
I20261118 02:55:04.431914  4332 tablet.cc:1759] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:04.438508  4333 tablet.cc:1759] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20261118 02:55:04.444600  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 1904140da78b4567a9decb250c6f74a3 (DEFAULT_TABLE table=default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.444877  3597 tablet_service.cc:1500] Processing CreateTablet for tablet 1904140da78b4567a9decb250c6f74a3 (DEFAULT_TABLE table=default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.445073  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1904140da78b4567a9decb250c6f74a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.445312  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1904140da78b4567a9decb250c6f74a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.448736  3880 tablet_service.cc:1500] Processing CreateTablet for tablet 1904140da78b4567a9decb250c6f74a3 (DEFAULT_TABLE table=default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.449203  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1904140da78b4567a9decb250c6f74a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.450577  3341 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_4f7796d5d5d54def843b971e66d15ea6" } new_table_name: "default.Table_79eb11797e3f4bc2bffe161096300f7a" modify_external_catalogs: true
I20261118 02:55:04.453258  4323 tablet_bootstrap.cc:492] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:04.455107  4323 tablet_bootstrap.cc:654] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.457552  3271 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:04.458042  3271 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:52040 (request call id 27) took 3723 ms. Trace:
I20261118 02:55:04.458174  3271 rpcz_store.cc:276] 1118 02:55:00.734475 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:02.105423 (+1370948us) service_pool.cc:224] Handling call
1118 02:55:03.452696 (+1347273us) catalog_manager.cc:2434] Looking up, locking, and authorizing table
1118 02:55:04.458000 (+1005304us) inbound_call.cc:177] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":814321,"Hive Metastore.run_cpu_time_us":198,"Hive Metastore.run_wall_time_us":189927}
I20261118 02:55:04.469877  4341 tablet_bootstrap.cc:492] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:04.471861  4341 tablet_bootstrap.cc:654] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.481155  4328 tablet_bootstrap.cc:492] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:04.483292  4328 tablet_bootstrap.cc:654] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.491559  4328 tablet_bootstrap.cc:492] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:04.491778  4328 ts_tablet_manager.cc:1449] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.011s	user 0.006s	sys 0.001s
I20261118 02:55:04.492671  4328 raft_consensus.cc:360] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.492899  4328 raft_consensus.cc:386] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.492985  4328 raft_consensus.cc:748] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.494174  4341 tablet_bootstrap.cc:492] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:04.494375  4341 ts_tablet_manager.cc:1449] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.025s	user 0.004s	sys 0.004s
I20261118 02:55:04.494832  3343 catalog_manager.cc:2215] created HMS catalog entry for table  [id=dd0f01caa36a4d3e9795daaf698b1ce1]
I20261118 02:55:04.495195  4328 consensus_queue.cc:261] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.496940  4328 ts_tablet_manager.cc:1480] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20261118 02:55:04.497500  4341 raft_consensus.cc:360] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.497733  4341 raft_consensus.cc:386] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.503237  4341 raft_consensus.cc:748] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.503867  4341 consensus_queue.cc:261] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.508061  4341 ts_tablet_manager.cc:1480] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.000s
I20261118 02:55:04.511054  4323 tablet_bootstrap.cc:492] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:04.511263  4323 ts_tablet_manager.cc:1449] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.000s
I20261118 02:55:04.512176  4323 raft_consensus.cc:360] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.512821  4323 raft_consensus.cc:386] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.512908  4323 raft_consensus.cc:748] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.513221  4323 consensus_queue.cc:261] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.516052  4323 ts_tablet_manager.cc:1480] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20261118 02:55:04.517192  3343 catalog_manager.cc:6393] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7] alter complete (version 1)
I20261118 02:55:04.524865  3734 tablet_service.cc:1500] Processing CreateTablet for tablet b129518ce23c45758ebcaa55410757d8 (DEFAULT_TABLE table=default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.525425  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b129518ce23c45758ebcaa55410757d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.525584  3597 tablet_service.cc:1500] Processing CreateTablet for tablet b129518ce23c45758ebcaa55410757d8 (DEFAULT_TABLE table=default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.526010  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b129518ce23c45758ebcaa55410757d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.528980  3880 tablet_service.cc:1500] Processing CreateTablet for tablet b129518ce23c45758ebcaa55410757d8 (DEFAULT_TABLE table=default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.529498  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b129518ce23c45758ebcaa55410757d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.535542  4323 tablet_bootstrap.cc:492] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:04.537458  4323 tablet_bootstrap.cc:654] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.542994  3343 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_0385cef6593c419eb426657c8cbef2bf" } new_table_name: "default.Table_496441d99faa4e2b8c7cd98e4a5ce745" modify_external_catalogs: true
I20261118 02:55:04.548003  4341 tablet_bootstrap.cc:492] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:04.549827  4341 tablet_bootstrap.cc:654] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.551963  2965 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task8fD3U4/build/asan/bin/kudu pid 3246
I20261118 02:55:04.556473  3307 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 02:55:04.560854  4323 tablet_bootstrap.cc:492] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:04.561024  4323 ts_tablet_manager.cc:1449] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.000s
I20261118 02:55:04.561877  4323 raft_consensus.cc:360] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.562078  4323 raft_consensus.cc:386] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.562173  4323 raft_consensus.cc:748] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.562422  4323 consensus_queue.cc:261] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.564137  4323 ts_tablet_manager.cc:1480] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.003s	user 0.003s	sys 0.000s
I20261118 02:55:04.578416  4328 tablet_bootstrap.cc:492] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:04.580322  3345 catalog_manager.cc:2215] created HMS catalog entry for table  [id=7140b942170b4483b0e01e32d5ba5e6d]
I20261118 02:55:04.584175  4328 tablet_bootstrap.cc:654] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.588632  3345 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48092 (request call id 15) took 1009 ms. Trace:
I20261118 02:55:04.588814  3345 rpcz_store.cc:276] 1118 02:55:03.579044 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.579563 (+   519us) service_pool.cc:224] Handling call
1118 02:55:04.276321 (+696758us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:04.276726 (+   405us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:04.580256 (+303530us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:04.588342 (+  8086us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:04.588462 (+   120us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:04.588586 (+   124us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:04.581268 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:04.581427 (+   159us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:04.581434 (+     7us) write_op.cc:435] Acquired schema lock
1118 02:55:04.581437 (+     3us) tablet.cc:670] Decoding operations
1118 02:55:04.582158 (+   721us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:04.582175 (+    17us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:04.582181 (+     6us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:04.582277 (+    96us) tablet.cc:709] Row locks acquired
1118 02:55:04.582280 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:04.582325 (+    45us) write_op.cc:270] Start()
1118 02:55:04.582364 (+    39us) write_op.cc:276] Timestamp: P: 1794970504582211 usec, L: 0
1118 02:55:04.582368 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:04.582610 (+   242us) log.cc:844] Serialized 432 byte log entry
1118 02:55:04.586943 (+  4333us) op_driver.cc:466] REPLICATION: finished
1118 02:55:04.587060 (+   117us) write_op.cc:301] APPLY: starting
1118 02:55:04.587079 (+    19us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:04.587126 (+    47us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:04.587163 (+    37us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:04.587299 (+   136us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:04.587333 (+    34us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:04.587341 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:04.587424 (+    83us) log.cc:844] Serialized 28 byte log entry
1118 02:55:04.587491 (+    67us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:04.587515 (+    24us) write_op.cc:454] Released schema lock
1118 02:55:04.587522 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:04.587594 (+    72us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:04.587599 (+     5us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":217932,"Hive Metastore.run_cpu_time_us":329,"Hive Metastore.run_wall_time_us":83894,"child_traces":[["op",{"apply.queue_time_us":65,"apply.run_cpu_time_us":525,"apply.run_wall_time_us":524,"num_ops":2,"prepare.queue_time_us":162,"prepare.run_cpu_time_us":936,"prepare.run_wall_time_us":1496,"raft.queue_time_us":515,"raft.run_cpu_time_us":686,"raft.run_wall_time_us":684,"replication_time_us":4540}]]}
I20261118 02:55:04.605414  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 2c92955aa65142f1bd20f4b8f3321500 (DEFAULT_TABLE table=default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.605541  3597 tablet_service.cc:1500] Processing CreateTablet for tablet 2c92955aa65142f1bd20f4b8f3321500 (DEFAULT_TABLE table=default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.605924  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c92955aa65142f1bd20f4b8f3321500. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.605921  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c92955aa65142f1bd20f4b8f3321500. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.609285  3880 tablet_service.cc:1500] Processing CreateTablet for tablet 2c92955aa65142f1bd20f4b8f3321500 (DEFAULT_TABLE table=default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:04.609778  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c92955aa65142f1bd20f4b8f3321500. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:04.612345  4323 tablet_bootstrap.cc:492] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:04.614121  4323 tablet_bootstrap.cc:654] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.618650  3345 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" } modify_external_catalogs: true
I20261118 02:55:04.618996  3345 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" } modify_external_catalogs: true
I20261118 02:55:04.627017  4323 tablet_bootstrap.cc:492] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:04.627230  4323 ts_tablet_manager.cc:1449] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.003s
I20261118 02:55:04.627993  4323 raft_consensus.cc:360] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.628306  4323 raft_consensus.cc:386] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.628427  4323 raft_consensus.cc:748] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.628737  4323 consensus_queue.cc:261] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.631083  4328 tablet_bootstrap.cc:492] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:04.632825  4323 ts_tablet_manager.cc:1480] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20261118 02:55:04.633181  4328 ts_tablet_manager.cc:1449] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.003s
I20261118 02:55:04.634070  4328 raft_consensus.cc:360] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.634274  4328 raft_consensus.cc:386] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.634378  4328 raft_consensus.cc:748] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.637547  4341 tablet_bootstrap.cc:492] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:04.637727  4341 ts_tablet_manager.cc:1449] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.090s	user 0.007s	sys 0.006s
I20261118 02:55:04.638218  4328 consensus_queue.cc:261] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.638607  4341 raft_consensus.cc:360] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.638782  4341 raft_consensus.cc:386] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.638867  4341 raft_consensus.cc:748] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.639106  4341 consensus_queue.cc:261] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.646626  4328 ts_tablet_manager.cc:1480] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.013s	user 0.001s	sys 0.001s
I20261118 02:55:04.650271  4341 ts_tablet_manager.cc:1480] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.001s
I20261118 02:55:04.650406  4328 tablet_bootstrap.cc:492] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:04.650604  4341 tablet_bootstrap.cc:492] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:04.652341  4341 tablet_bootstrap.cc:654] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.653347  4360 raft_consensus.cc:499] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:04.653501  4360 raft_consensus.cc:521] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.654232  4360 leader_election.cc:290] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.655382  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b610080fc91745c7be260699946d5681" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:04.655633  3616 raft_consensus.cc:2490] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:04.656010  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b610080fc91745c7be260699946d5681" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:04.656296  3902 raft_consensus.cc:2490] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:04.657063  3690 leader_election.cc:304] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.657759  4360 raft_consensus.cc:2826] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:04.657883  4360 raft_consensus.cc:499] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:04.657971  4360 raft_consensus.cc:3096] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.661325  4328 tablet_bootstrap.cc:654] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:04.662608  4360 raft_consensus.cc:521] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.663757  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b610080fc91745c7be260699946d5681" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:04.663996  3616 raft_consensus.cc:3096] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.667074  3616 raft_consensus.cc:2490] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:04.667618  3691 leader_election.cc:304] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.667917  4300 raft_consensus.cc:2826] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:04.671132  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b610080fc91745c7be260699946d5681" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:04.671389  3902 raft_consensus.cc:3096] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.674564  3902 raft_consensus.cc:2490] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:04.675848  4360 leader_election.cc:290] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.681222  4328 tablet_bootstrap.cc:492] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:04.681438  4328 ts_tablet_manager.cc:1449] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.004s
I20261118 02:55:04.682225  4328 raft_consensus.cc:360] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.682440  4328 raft_consensus.cc:386] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.682519  4328 raft_consensus.cc:748] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.682826  4328 consensus_queue.cc:261] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.683494  4328 ts_tablet_manager.cc:1480] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.001s
I20261118 02:55:04.691319  4300 raft_consensus.cc:705] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:04.691655  4300 consensus_queue.cc:238] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:04.695037  4341 tablet_bootstrap.cc:492] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:04.703747  4360 consensus_queue.cc:1049] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.708168  4300 consensus_queue.cc:1049] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.724174  4263 raft_consensus.cc:499] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:04.724373  4263 raft_consensus.cc:521] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.724961  4263 leader_election.cc:290] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.725039  4341 ts_tablet_manager.cc:1449] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.074s	user 0.004s	sys 0.000s
I20261118 02:55:04.726269  3753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b129518ce23c45758ebcaa55410757d8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:04.726512  3753 raft_consensus.cc:2490] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:04.727325  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b129518ce23c45758ebcaa55410757d8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:04.727535  3902 raft_consensus.cc:2490] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:04.728065  3552 leader_election.cc:304] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.728480  4341 raft_consensus.cc:360] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.728674  4341 raft_consensus.cc:386] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:04.728690  4265 raft_consensus.cc:2826] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:04.728730  4341 raft_consensus.cc:748] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:04.728797  4265 raft_consensus.cc:499] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:04.728873  4265 raft_consensus.cc:3096] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.728973  4341 consensus_queue.cc:261] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.730585  4341 ts_tablet_manager.cc:1480] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.001s
I20261118 02:55:04.731884  4265 raft_consensus.cc:521] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.735399  4265 leader_election.cc:290] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.736183  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b129518ce23c45758ebcaa55410757d8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:04.736174  3753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b129518ce23c45758ebcaa55410757d8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:04.736389  3753 raft_consensus.cc:3096] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.736858  3902 raft_consensus.cc:3096] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.739348  3753 raft_consensus.cc:2490] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:04.739645  3902 raft_consensus.cc:2490] T b129518ce23c45758ebcaa55410757d8 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:04.740029  3553 leader_election.cc:304] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:04.743608  4265 raft_consensus.cc:2826] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:04.744136  4265 raft_consensus.cc:705] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:04.744565  4265 consensus_queue.cc:238] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.768595  4263 consensus_queue.cc:1049] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.777444  4384 consensus_queue.cc:1049] T b129518ce23c45758ebcaa55410757d8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.793957  4384 raft_consensus.cc:499] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:04.794260  4384 raft_consensus.cc:521] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.795543  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1904140da78b4567a9decb250c6f74a3" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:04.795845  3751 raft_consensus.cc:2490] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:04.795965  4384 leader_election.cc:290] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.796357  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1904140da78b4567a9decb250c6f74a3" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:04.796380  3552 leader_election.cc:304] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.796587  3901 raft_consensus.cc:2490] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:04.796798  4265 raft_consensus.cc:2826] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:04.796900  4265 raft_consensus.cc:499] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:04.796975  4265 raft_consensus.cc:3096] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.799749  4265 raft_consensus.cc:521] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.800417  4265 leader_election.cc:290] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.801316  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1904140da78b4567a9decb250c6f74a3" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:04.801558  3751 raft_consensus.cc:3096] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.804633  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "1904140da78b4567a9decb250c6f74a3" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:04.804874  3901 raft_consensus.cc:3096] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.809340  3751 raft_consensus.cc:2490] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:04.810060  3901 raft_consensus.cc:2490] T 1904140da78b4567a9decb250c6f74a3 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:04.810653  3553 leader_election.cc:304] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:04.810959  4387 raft_consensus.cc:2826] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:04.811206  4387 raft_consensus.cc:705] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:04.811496  4387 consensus_queue.cc:238] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.839188  4263 consensus_queue.cc:1049] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.840451  4265 consensus_queue.cc:1049] T 1904140da78b4567a9decb250c6f74a3 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.883309  3270 catalog_manager.cc:2225] deleted orphaned HMS catalog entry for table  [id=baa54e879f6246509f1c5ca1876f0b25]
I20261118 02:55:04.883800  3270 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:52040 (request call id 30) took 4046 ms. Trace:
I20261118 02:55:04.883934  3270 rpcz_store.cc:276] 1118 02:55:00.836975 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:02.367849 (+1530874us) service_pool.cc:224] Handling call
1118 02:55:03.447690 (+1079841us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:03.448571 (+   881us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:04.267317 (+818746us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:04.268119 (+   802us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:04.268123 (+     4us) write_op.cc:454] Released schema lock
1118 02:55:04.268355 (+   232us) catalog_manager.cc:2222] Rolling back HMS table creation
1118 02:55:04.883754 (+615399us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:04.268000 (+     0us) op_driver.cc:391] HandleFailure(Illegal state: replica 26c6b20fa203404a88463ea27bdd3c7d is not leader of this config: current role FOLLOWER)
1118 02:55:04.268014 (+    14us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:04.268022 (+     8us) write_op.cc:454] Released schema lock
1118 02:55:04.268026 (+     4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":869586,"Hive Metastore.run_cpu_time_us":473,"Hive Metastore.run_wall_time_us":563445,"child_traces":[["op",{}]]}
I20261118 02:55:04.884406  3273 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
W20261118 02:55:04.886116  3956 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.229.125:38339)
I20261118 02:55:04.889158  2965 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task8fD3U4/build/asan/bin/kudu with pid 3246
I20261118 02:55:04.893725  3342 catalog_manager.cc:3543] renamed table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6] in HMS: new name default.table_5091d82298ca44bea5335d4989eb02d8
W20261118 02:55:04.921594  3956 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.2.229.125:38339)
W20261118 02:55:04.923596  3333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.229.126:38025 (error 108) [suppressed 5 similar messages]
W20261118 02:55:04.934947  3333 consensus_peers.cc:597] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b -> Peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Couldn't send request to peer 26c6b20fa203404a88463ea27bdd3c7d. Status: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:04.936760  4384 raft_consensus.cc:499] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:04.936966  4384 raft_consensus.cc:521] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.937793  4384 leader_election.cc:290] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.938542  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2c92955aa65142f1bd20f4b8f3321500" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:04.938555  3753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2c92955aa65142f1bd20f4b8f3321500" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:04.938763  3902 raft_consensus.cc:2490] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:04.938771  3753 raft_consensus.cc:2490] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:04.939054  3339 catalog_manager.cc:2772] deleted HMS catalog entry for table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:04.939286  3552 leader_election.cc:304] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.939589  4384 raft_consensus.cc:2826] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:04.939703  4384 raft_consensus.cc:499] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:04.939790  4384 raft_consensus.cc:3096] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.942586  4384 raft_consensus.cc:521] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.943732  3753 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2c92955aa65142f1bd20f4b8f3321500" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:04.943993  3753 raft_consensus.cc:3096] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.947000  3753 raft_consensus.cc:2490] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:04.947556  3552 leader_election.cc:304] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:04.947909  4263 raft_consensus.cc:2826] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:04.948065  4263 raft_consensus.cc:705] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:04.948397  4263 consensus_queue.cc:238] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:04.958894  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "2c92955aa65142f1bd20f4b8f3321500" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:04.959163  3902 raft_consensus.cc:3096] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:04.962152  3902 raft_consensus.cc:2490] T 2c92955aa65142f1bd20f4b8f3321500 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:04.963047  4384 leader_election.cc:290] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:04.978030  4387 consensus_queue.cc:1049] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:04.981541  4387 consensus_queue.cc:1049] T 2c92955aa65142f1bd20f4b8f3321500 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:05.040940  3346 catalog_manager.cc:3543] renamed table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73] in HMS: new name default.table_324a2600fef947748ef03e27f7cb7e09
I20261118 02:55:05.073117  3338 catalog_manager.cc:2215] created HMS catalog entry for table  [id=00713c8ca8db4bf9a860d32614f99190]
I20261118 02:55:05.078691  3338 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48092 (request call id 23) took 1396 ms. Trace:
I20261118 02:55:05.078833  3338 rpcz_store.cc:276] 1118 02:55:03.682580 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.682977 (+   397us) service_pool.cc:224] Handling call
1118 02:55:04.277647 (+594670us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:04.278070 (+   423us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:05.073076 (+795006us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:05.078330 (+  5254us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:05.078458 (+   128us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:05.078602 (+   144us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:05.073924 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:05.074068 (+   144us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:05.074074 (+     6us) write_op.cc:435] Acquired schema lock
1118 02:55:05.074075 (+     1us) tablet.cc:670] Decoding operations
1118 02:55:05.074159 (+    84us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:05.074171 (+    12us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:05.074175 (+     4us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:05.074242 (+    67us) tablet.cc:709] Row locks acquired
1118 02:55:05.074244 (+     2us) write_op.cc:260] PREPARE: finished
1118 02:55:05.074279 (+    35us) write_op.cc:270] Start()
1118 02:55:05.074310 (+    31us) write_op.cc:276] Timestamp: P: 1794970505074170 usec, L: 0
1118 02:55:05.074312 (+     2us) op_driver.cc:350] REPLICATION: starting
1118 02:55:05.074493 (+   181us) log.cc:844] Serialized 432 byte log entry
1118 02:55:05.077540 (+  3047us) op_driver.cc:466] REPLICATION: finished
1118 02:55:05.077708 (+   168us) write_op.cc:301] APPLY: starting
1118 02:55:05.077726 (+    18us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:05.077778 (+    52us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:05.077809 (+    31us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:05.077946 (+   137us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:05.077984 (+    38us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:05.077992 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:05.078117 (+   125us) log.cc:844] Serialized 28 byte log entry
1118 02:55:05.078202 (+    85us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:05.078226 (+    24us) write_op.cc:454] Released schema lock
1118 02:55:05.078234 (+     8us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:05.078316 (+    82us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:05.078320 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":762713,"Hive Metastore.run_cpu_time_us":341,"Hive Metastore.run_wall_time_us":32084,"child_traces":[["op",{"apply.queue_time_us":114,"apply.run_cpu_time_us":599,"apply.run_wall_time_us":598,"num_ops":2,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":694,"prepare.run_wall_time_us":694,"raft.queue_time_us":93,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":332,"replication_time_us":3195}]]}
I20261118 02:55:05.103708  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 0e901a03b90e4c978eb98be87163cfe5 (DEFAULT_TABLE table=default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:05.103943  3880 tablet_service.cc:1500] Processing CreateTablet for tablet 0e901a03b90e4c978eb98be87163cfe5 (DEFAULT_TABLE table=default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:05.104224  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e901a03b90e4c978eb98be87163cfe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:05.104449  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e901a03b90e4c978eb98be87163cfe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:05.105386  3338 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48092:
table { table_name: "default.Table_2e3917ff98f2474984580e61caa9c17b" } new_table_name: "default.Table_3c225da1f7fa435083909e053ec9751a" modify_external_catalogs: true
I20261118 02:55:05.108930  3597 tablet_service.cc:1500] Processing CreateTablet for tablet 0e901a03b90e4c978eb98be87163cfe5 (DEFAULT_TABLE table=default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:05.109357  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e901a03b90e4c978eb98be87163cfe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:05.112860  4341 tablet_bootstrap.cc:492] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:05.114666  4341 tablet_bootstrap.cc:654] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:05.116966  4323 tablet_bootstrap.cc:492] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:05.118590  4323 tablet_bootstrap.cc:654] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:05.119942  4328 tablet_bootstrap.cc:492] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:05.121654  4328 tablet_bootstrap.cc:654] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:05.125878  4341 tablet_bootstrap.cc:492] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:05.126117  4341 ts_tablet_manager.cc:1449] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.013s	user 0.005s	sys 0.000s
I20261118 02:55:05.126958  4341 raft_consensus.cc:360] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.127720  4341 raft_consensus.cc:386] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:05.128005  4341 raft_consensus.cc:748] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:05.128612  4341 consensus_queue.cc:261] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.129287  3340 catalog_manager.cc:2215] created HMS catalog entry for table  [id=01b05aad48954b8fb2aa6d7b31a6578c]
I20261118 02:55:05.131323  4328 tablet_bootstrap.cc:492] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:05.131340  4323 tablet_bootstrap.cc:492] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:05.131479  4323 ts_tablet_manager.cc:1449] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.007s
I20261118 02:55:05.131623  4328 ts_tablet_manager.cc:1449] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.012s	user 0.004s	sys 0.000s
I20261118 02:55:05.133008  4328 raft_consensus.cc:360] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.133245  4328 raft_consensus.cc:386] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:05.134723  4323 raft_consensus.cc:360] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.134918  4323 raft_consensus.cc:386] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:05.135020  4323 raft_consensus.cc:748] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:05.135336  4323 consensus_queue.cc:261] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.136013  4323 ts_tablet_manager.cc:1480] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.001s
I20261118 02:55:05.137785  3340 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:48092 (request call id 11) took 1577 ms. Trace:
I20261118 02:55:05.137940  3340 rpcz_store.cc:276] 1118 02:55:03.560333 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:03.560379 (+    46us) service_pool.cc:224] Handling call
1118 02:55:04.277424 (+717045us) catalog_manager.cc:2142] Acquired catalog manager lock
1118 02:55:04.278168 (+   744us) catalog_manager.cc:2188] Created new table and tablet info
1118 02:55:05.129224 (+851056us) catalog_manager.cc:2214] Created new table in HMS catalog
1118 02:55:05.137471 (+  8247us) catalog_manager.cc:2248] Wrote table and tablets to system table
1118 02:55:05.137615 (+   144us) catalog_manager.cc:2282] Inserted table and tablets into CatalogManager maps
1118 02:55:05.137735 (+   120us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:05.130158 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:05.130287 (+   129us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:05.130294 (+     7us) write_op.cc:435] Acquired schema lock
1118 02:55:05.130296 (+     2us) tablet.cc:670] Decoding operations
1118 02:55:05.130394 (+    98us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:05.130408 (+    14us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:05.130412 (+     4us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:05.130501 (+    89us) tablet.cc:709] Row locks acquired
1118 02:55:05.130504 (+     3us) write_op.cc:260] PREPARE: finished
1118 02:55:05.130542 (+    38us) write_op.cc:270] Start()
1118 02:55:05.130577 (+    35us) write_op.cc:276] Timestamp: P: 1794970505130524 usec, L: 0
1118 02:55:05.130581 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:05.130774 (+   193us) log.cc:844] Serialized 432 byte log entry
1118 02:55:05.136311 (+  5537us) op_driver.cc:466] REPLICATION: finished
1118 02:55:05.136956 (+   645us) write_op.cc:301] APPLY: starting
1118 02:55:05.136974 (+    18us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:05.137013 (+    39us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:05.137046 (+    33us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:05.137158 (+   112us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:05.137183 (+    25us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
1118 02:55:05.137188 (+     5us) write_op.cc:312] APPLY: finished
1118 02:55:05.137258 (+    70us) log.cc:844] Serialized 28 byte log entry
1118 02:55:05.137319 (+    61us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:05.137339 (+    20us) write_op.cc:454] Released schema lock
1118 02:55:05.137346 (+     7us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:05.137413 (+    67us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:05.137417 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":794789,"Hive Metastore.run_cpu_time_us":277,"Hive Metastore.run_wall_time_us":56109,"child_traces":[["op",{"apply.queue_time_us":577,"apply.run_cpu_time_us":456,"apply.run_wall_time_us":457,"num_ops":2,"prepare.queue_time_us":202,"prepare.run_cpu_time_us":734,"prepare.run_wall_time_us":735,"raft.queue_time_us":1793,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":346,"replication_time_us":5697}]]}
I20261118 02:55:05.133363  4328 raft_consensus.cc:748] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:05.139240  4328 consensus_queue.cc:261] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
F20261118 02:55:05.142462  3527 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 15 received events: NotificationEvent(eventId=21, eventTime=1794970503, eventType=CREATE_TABLE, dbName=default, tableName=table_f0ae490a0df64e1097e144c7a370df61, message={"server":"","servicePrincipal":"","db":"default","table":"table_f0ae490a0df64e1097e144c7a370df61","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f0ae490a0df64e1097e144c7a370df61\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970503},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_f0ae490a0df64e1097e144c7a370df61\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f0ae490a0df64e1097e144c7a370df61\",\"kudu.table_id\":\"a59b9425c01049269883cf7767936c67\",\"transient_lastDdlTime\":\"1794970503\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1794970503, eventType=DROP_TABLE, dbName=default, tableName=table_c113c2b57e17479e92e241234a241af7, message={"server":"","servicePrincipal":"","db":"default","table":"table_c113c2b57e17479e92e241234a241af7","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c113c2b57e17479e92e241234a241af7\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970500},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_c113c2b57e17479e92e241234a241af7\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c113c2b57e17479e92e241234a241af7\",\"kudu.table_id\":\"da0cb25ccfed426ca61f0dd9f93d30ec\",\"transient_lastDdlTime\":\"1794970500\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20261118 02:55:05.144402  4328 ts_tablet_manager.cc:1480] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.013s	user 0.000s	sys 0.002s
    @     0x7f72ad19bd5d  google::LogMessage::Fail() at ??:0
I20261118 02:55:05.145990  4341 ts_tablet_manager.cc:1480] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.001s
I20261118 02:55:05.146613  3340 catalog_manager.cc:5840] T b129518ce23c45758ebcaa55410757d8 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } }
I20261118 02:55:05.147181  3340 catalog_manager.cc:5840] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66). New cstate: current_term: 1 leader_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } health_report { overall_health: HEALTHY } } }
I20261118 02:55:05.147773  3340 catalog_manager.cc:5840] T 2c92955aa65142f1bd20f4b8f3321500 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } }
I20261118 02:55:05.147877  3881 tablet_service.cc:1500] Processing CreateTablet for tablet 8bed657d0da64d95851f8a86216b6c2c (DEFAULT_TABLE table=default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:05.148326  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bed657d0da64d95851f8a86216b6c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:05.148185  3340 catalog_manager.cc:5840] T 1904140da78b4567a9decb250c6f74a3 P 99a39b1d9d814deeb3a8a644e9b392ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65). New cstate: current_term: 1 leader_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } }
    @     0x7f72ad19fb23  google::LogMessage::SendToLog() at ??:0
    @     0x7f72ad19b75c  google::LogMessage::Flush() at ??:0
    @     0x7f72ad19b9a1  google::LogMessage::~LogMessage() at ??:0
I20261118 02:55:05.153110  3734 tablet_service.cc:1500] Processing CreateTablet for tablet 8bed657d0da64d95851f8a86216b6c2c (DEFAULT_TABLE table=default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:05.153513  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bed657d0da64d95851f8a86216b6c2c. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f72c261b9a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 02:55:05.159298  3597 tablet_service.cc:1500] Processing CreateTablet for tablet 8bed657d0da64d95851f8a86216b6c2c (DEFAULT_TABLE table=default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:05.159722  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bed657d0da64d95851f8a86216b6c2c. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f72c261a251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 02:55:05.166616  4328 tablet_bootstrap.cc:492] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:05.168407  4328 tablet_bootstrap.cc:654] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
    @     0x7f72c2621ecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 02:55:05.169085  4323 tablet_bootstrap.cc:492] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:05.170626  4341 tablet_bootstrap.cc:492] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:05.170773  4323 tablet_bootstrap.cc:654] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:05.171552  3340 master_service.cc:965] ReplaceTablet: received request to replace tablet 489beea71ae74236addee59c153cfee7 from {username='slave'} at 127.0.0.1:48092
    @     0x7f72c2621d49  std::_Function_handler<>::_M_invoke() at ??:0
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 02:55:05.173688  4341 tablet_bootstrap.cc:654] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:05.175356  4328 tablet_bootstrap.cc:492] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:05.175529  4328 ts_tablet_manager.cc:1449] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.009s	user 0.003s	sys 0.000s
    @     0x7f72ae5e4dc9  kudu::Thread::SuperviseThread() at ??:0
I20261118 02:55:05.176349  4328 raft_consensus.cc:360] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.176560  4328 raft_consensus.cc:386] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:05.176623  4328 raft_consensus.cc:748] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:05.176872  4328 consensus_queue.cc:261] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.178706  4328 ts_tablet_manager.cc:1480] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:05.179394  3340 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet 489beea71ae74236addee59c153cfee7
I20261118 02:55:05.180143  3595 tablet_service.cc:1547] Processing DeleteTablet for tablet 489beea71ae74236addee59c153cfee7 with delete_type TABLET_DATA_DELETED (replaced by tablet ccf78258c9634540a178855a142f4d4b) from {username='slave'} at 127.0.0.1:53002
I20261118 02:55:05.180500  3340 catalog_manager.cc:6936] ReplaceTablet: tablet 489beea71ae74236addee59c153cfee7 deleted and replaced by tablet ccf78258c9634540a178855a142f4d4b
I20261118 02:55:05.180761  3734 tablet_service.cc:1547] Processing DeleteTablet for tablet 489beea71ae74236addee59c153cfee7 with delete_type TABLET_DATA_DELETED (replaced by tablet ccf78258c9634540a178855a142f4d4b) from {username='slave'} at 127.0.0.1:57892
I20261118 02:55:05.180630  3340 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:48092 (request call id 31) took 1320 ms. Trace:
I20261118 02:55:05.182221  3880 tablet_service.cc:1547] Processing DeleteTablet for tablet 489beea71ae74236addee59c153cfee7 with delete_type TABLET_DATA_DELETED (replaced by tablet ccf78258c9634540a178855a142f4d4b) from {username='slave'} at 127.0.0.1:50250
I20261118 02:55:05.182708  3340 rpcz_store.cc:276] 1118 02:55:03.860229 (+     0us) service_pool.cc:167] Inserting onto call queue
1118 02:55:05.170378 (+1310149us) service_pool.cc:224] Handling call
1118 02:55:05.180578 (+ 10200us) inbound_call.cc:177] Queueing success response
Related trace 'op':
1118 02:55:05.172773 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
1118 02:55:05.172891 (+   118us) write_op.cc:432] Acquiring schema lock in shared mode
1118 02:55:05.172898 (+     7us) write_op.cc:435] Acquired schema lock
1118 02:55:05.172900 (+     2us) tablet.cc:670] Decoding operations
1118 02:55:05.173012 (+   112us) write_op.cc:620] Acquiring the partition lock for write op
1118 02:55:05.173026 (+    14us) write_op.cc:641] Partition lock acquired for write op
1118 02:55:05.173031 (+     5us) tablet.cc:693] Acquiring locks for 2 operations
1118 02:55:05.173107 (+    76us) tablet.cc:709] Row locks acquired
1118 02:55:05.173109 (+     2us) write_op.cc:260] PREPARE: finished
1118 02:55:05.173144 (+    35us) write_op.cc:270] Start()
1118 02:55:05.173176 (+    32us) write_op.cc:276] Timestamp: P: 1794970505173118 usec, L: 0
1118 02:55:05.173180 (+     4us) op_driver.cc:350] REPLICATION: starting
1118 02:55:05.173350 (+   170us) log.cc:844] Serialized 658 byte log entry
1118 02:55:05.176965 (+  3615us) op_driver.cc:466] REPLICATION: finished
1118 02:55:05.177137 (+   172us) write_op.cc:301] APPLY: starting
1118 02:55:05.177153 (+    16us) tablet.cc:1402] starting BulkCheckPresence
1118 02:55:05.177203 (+    50us) tablet.cc:1405] finished BulkCheckPresence
1118 02:55:05.177232 (+    29us) tablet.cc:1407] starting ApplyRowOperation cycle
1118 02:55:05.177334 (+   102us) tablet.cc:1418] finished ApplyRowOperation cycle
1118 02:55:05.177383 (+    49us) tablet_metrics.cc:626] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
1118 02:55:05.177391 (+     8us) write_op.cc:312] APPLY: finished
1118 02:55:05.177461 (+    70us) log.cc:844] Serialized 28 byte log entry
1118 02:55:05.177525 (+    64us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:05.177547 (+    22us) write_op.cc:454] Released schema lock
1118 02:55:05.177553 (+     6us) write_op.cc:341] FINISH: Updating metrics
1118 02:55:05.177634 (+    81us) write_op.cc:489] Releasing partition, row and schema locks
1118 02:55:05.177638 (+     4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":119,"apply.run_cpu_time_us":488,"apply.run_wall_time_us":488,"num_ops":2,"prepare.queue_time_us":46,"prepare.run_cpu_time_us":702,"prepare.run_wall_time_us":702,"raft.queue_time_us":560,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":302,"replication_time_us":3754}]]}
I20261118 02:55:05.189301  3340 master_service.cc:965] ReplaceTablet: received request to replace tablet a013d180f2cd430ab1ba30b39fd27591 from {username='slave'} at 127.0.0.1:48092
I20261118 02:55:05.189515  4399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:05.189739  4399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:05.189939  3735 tablet_service.cc:1500] Processing CreateTablet for tablet ccf78258c9634540a178855a142f4d4b (DEFAULT_TABLE table=default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:05.190398  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf78258c9634540a178855a142f4d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:05.190337  3594 tablet_service.cc:1500] Processing CreateTablet for tablet ccf78258c9634540a178855a142f4d4b (DEFAULT_TABLE table=default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:05.190773  3594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf78258c9634540a178855a142f4d4b. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f72b1cbf6db  start_thread at ??:0
    @     0x7f72aa72f61f  clone at ??:0
I20261118 02:55:05.200424  4323 tablet_bootstrap.cc:492] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:05.200577  4410 tablet_replica.cc:333] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:05.200605  4323 ts_tablet_manager.cc:1449] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.032s	user 0.003s	sys 0.000s
I20261118 02:55:05.200768  4410 raft_consensus.cc:2261] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:05.201126  4410 raft_consensus.cc:2290] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:05.201323  4323 raft_consensus.cc:360] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.201519  4323 raft_consensus.cc:386] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:05.201596  4323 raft_consensus.cc:748] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:05.201702  4410 ts_tablet_manager.cc:1962] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:05.201846  4323 consensus_queue.cc:261] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.202301  4323 ts_tablet_manager.cc:1480] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:05.202649  4323 tablet_bootstrap.cc:492] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:05.206444  4323 tablet_bootstrap.cc:654] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:05.207084  4408 tablet_replica.cc:333] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:05.207288  4408 raft_consensus.cc:2261] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:05.207480  4408 raft_consensus.cc:2290] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:05.208034  4408 ts_tablet_manager.cc:1962] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:05.211724  4323 tablet_bootstrap.cc:492] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:05.211874  4323 ts_tablet_manager.cc:1449] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.009s	user 0.004s	sys 0.000s
I20261118 02:55:05.212636  4409 tablet_replica.cc:333] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:05.212711  4323 raft_consensus.cc:360] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:05.212854  4409 raft_consensus.cc:2261] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:05.212908  4323 raft_consensus.cc:386] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:05.212996  4323 raft_consensus.cc:748] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:05.213106  4409 raft_consensus.cc:2290] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:05.213279  4323 consensus_queue.cc:261] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:05.213687  4409 ts_tablet_manager.cc:1962] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:05.215392  4323 ts_tablet_manager.cc:1480] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:05.216957  4341 tablet_bootstrap.cc:492] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:05.217123  4341 ts_tablet_manager.cc:1449] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.015s
I20261118 02:55:05.217851  4341 raft_consensus.cc:360] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.218035  4341 raft_consensus.cc:386] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:05.218169  4341 raft_consensus.cc:748] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:05.218425  4341 consensus_queue.cc:261] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.218950  4341 ts_tablet_manager.cc:1480] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 02:55:05.220050  4341 tablet_bootstrap.cc:492] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:05.221827  4341 tablet_bootstrap.cc:654] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:05.223308  4410 ts_tablet_manager.cc:1975] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:05.223424  4410 log.cc:1199] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/489beea71ae74236addee59c153cfee7
I20261118 02:55:05.223821  4410 ts_tablet_manager.cc:1996] T 489beea71ae74236addee59c153cfee7 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:05.227204  4408 ts_tablet_manager.cc:1975] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:05.227327  4408 log.cc:1199] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/489beea71ae74236addee59c153cfee7
I20261118 02:55:05.227715  4408 ts_tablet_manager.cc:1996] T 489beea71ae74236addee59c153cfee7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:05.228267  4341 tablet_bootstrap.cc:492] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:05.228411  4341 ts_tablet_manager.cc:1449] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.008s	user 0.002s	sys 0.005s
I20261118 02:55:05.229288  4341 raft_consensus.cc:360] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:05.229461  4341 raft_consensus.cc:386] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:05.229548  4341 raft_consensus.cc:748] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:05.229779  4341 consensus_queue.cc:261] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:05.230417  4341 ts_tablet_manager.cc:1480] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:05.230805  4409 ts_tablet_manager.cc:1975] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:05.230931  4409 log.cc:1199] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/489beea71ae74236addee59c153cfee7
I20261118 02:55:05.231369  4409 ts_tablet_manager.cc:1996] T 489beea71ae74236addee59c153cfee7 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:05.262112  4379 raft_consensus.cc:499] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:05.262317  4379 raft_consensus.cc:521] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.263139  4379 leader_election.cc:290] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:05.263607  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8bed657d0da64d95851f8a86216b6c2c" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:05.263814  3901 raft_consensus.cc:2490] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:05.263940  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8bed657d0da64d95851f8a86216b6c2c" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:05.264169  3616 raft_consensus.cc:2490] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:05.264343  3690 leader_election.cc:304] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:05.264626  4379 raft_consensus.cc:2826] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:05.264745  4379 raft_consensus.cc:499] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:05.264823  4379 raft_consensus.cc:3096] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:05.266711  4379 raft_consensus.cc:521] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.267699  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8bed657d0da64d95851f8a86216b6c2c" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:05.267707  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8bed657d0da64d95851f8a86216b6c2c" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:05.267884  3616 raft_consensus.cc:3096] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:05.267896  3901 raft_consensus.cc:3096] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:05.270753  3901 raft_consensus.cc:2490] T 8bed657d0da64d95851f8a86216b6c2c P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:05.270768  3616 raft_consensus.cc:2490] T 8bed657d0da64d95851f8a86216b6c2c P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:05.271226  3691 leader_election.cc:304] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:05.271725  4379 leader_election.cc:290] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:05.271934  4379 raft_consensus.cc:2826] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:05.272045  4379 raft_consensus.cc:705] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:05.272369  4379 consensus_queue.cc:238] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.285882  4421 consensus_queue.cc:1049] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:05.288618  4423 consensus_queue.cc:1049] T 8bed657d0da64d95851f8a86216b6c2c P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 02:55:05.426714  3691 connection.cc:573] server connection from 127.0.0.1:57892 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:05.426754  3553 connection.cc:573] server connection from 127.0.0.1:53002 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:05.426754  3835 connection.cc:573] server connection from 127.0.0.1:50250 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:05.427068  4137 connection.cc:573] server connection from 127.0.0.1:34230 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:05.427281  3836 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:05.427372  3689 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:05.428056  3552 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:05.428491  3955 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:05.428947  3955 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.229.125:38339)
W20261118 02:55:05.430042  3955 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.2.229.125:38339)
I20261118 02:55:05.642093  4421 raft_consensus.cc:499] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:05.642302  4421 raft_consensus.cc:521] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.642930  4421 leader_election.cc:290] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:05.643451  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0e901a03b90e4c978eb98be87163cfe5" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:05.643455  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0e901a03b90e4c978eb98be87163cfe5" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:05.643636  3616 raft_consensus.cc:2490] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:05.643646  3901 raft_consensus.cc:2490] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:05.644119  3691 leader_election.cc:304] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:05.644411  4421 raft_consensus.cc:2826] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:05.644507  4421 raft_consensus.cc:499] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:05.644608  4421 raft_consensus.cc:3096] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:05.646481  4421 raft_consensus.cc:521] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.647061  4421 leader_election.cc:290] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:05.647534  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0e901a03b90e4c978eb98be87163cfe5" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:05.647552  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "0e901a03b90e4c978eb98be87163cfe5" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:05.647730  3616 raft_consensus.cc:3096] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:05.647730  3901 raft_consensus.cc:3096] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:05.650602  3616 raft_consensus.cc:2490] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:05.650617  3901 raft_consensus.cc:2490] T 0e901a03b90e4c978eb98be87163cfe5 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:05.651042  3691 leader_election.cc:304] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:05.651304  4421 raft_consensus.cc:2826] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:05.651455  4421 raft_consensus.cc:705] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:05.651746  4421 consensus_queue.cc:238] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:05.654352  4423 consensus_queue.cc:1049] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:05.654570  4379 consensus_queue.cc:1049] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:05.698751  4437 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:05.698905  4437 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:05.699532  4437 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:05.700556  4139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:05.701692  4138 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:05.701833  4139 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:05.701985  4139 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:05.702286  4437 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20261118 02:55:05.776943  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.126:38025
--webserver_interface=127.2.229.126
--webserver_port=44955
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 02:55:05.836552  4421 raft_consensus.cc:499] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:05.836755  4421 raft_consensus.cc:521] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:05.837512  4421 leader_election.cc:290] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:05.838105  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ccf78258c9634540a178855a142f4d4b" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:05.838363  3616 raft_consensus.cc:2490] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
W20261118 02:55:05.838675  3690 leader_election.cc:343] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: ccf78258c9634540a178855a142f4d4b
I20261118 02:55:05.838943  3691 leader_election.cc:304] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 3f1973986ec1466eab5bec01dd81654f
I20261118 02:55:05.839244  4421 raft_consensus.cc:2826] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:05.839365  4421 raft_consensus.cc:499] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:05.839455  4421 raft_consensus.cc:3096] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:05.842037  4421 raft_consensus.cc:521] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:05.842864  4421 leader_election.cc:290] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:05.843354  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ccf78258c9634540a178855a142f4d4b" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:05.843525  3616 raft_consensus.cc:3096] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
W20261118 02:55:05.843816  3690 leader_election.cc:343] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: ccf78258c9634540a178855a142f4d4b
I20261118 02:55:05.845495  3616 raft_consensus.cc:2490] T ccf78258c9634540a178855a142f4d4b P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:05.845882  3691 leader_election.cc:304] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 3f1973986ec1466eab5bec01dd81654f
I20261118 02:55:05.846185  4421 raft_consensus.cc:2826] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:05.846311  4421 raft_consensus.cc:705] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:05.846582  4421 consensus_queue.cc:238] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
W20261118 02:55:05.850759  3690 consensus_peers.cc:597] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Couldn't send request to peer 3f1973986ec1466eab5bec01dd81654f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ccf78258c9634540a178855a142f4d4b. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:05.850984  4421 consensus_queue.cc:1049] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 02:55:06.139664  4441 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:55:06.139981  4441 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:55:06.140116  4441 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:55:06.148783  4441 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:55:06.148885  4441 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:55:06.148962  4441 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:55:06.149024  4441 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:55:06.159363  4441 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.126:38025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.229.126
--webserver_port=44955
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:55:06.161120  4441 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:55:06.163302  4441 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:55:06.172415  4449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:06.172385  4448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:06.172384  4451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:06.173213  4441 server_base.cc:1061] running on GCE node
I20261118 02:55:06.173975  4441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:55:06.175623  4441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:55:06.176844  4441 hybrid_clock.cc:648] HybridClock initialized: now 1794970506176791 us; error 45 us; skew 500 ppm
I20261118 02:55:06.177234  4441 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:55:06.179368  4441 webserver.cc:577] Webserver started at http://127.2.229.126:44955/ using document root <none> and password file <none>
I20261118 02:55:06.180065  4441 fs_manager.cc:362] Metadata directory not provided
I20261118 02:55:06.180224  4441 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:55:06.185964  4441 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.001s	sys 0.003s
I20261118 02:55:06.189163  4457 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:55:06.190706  4441 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20261118 02:55:06.190864  4441 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
uuid: "26c6b20fa203404a88463ea27bdd3c7d"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:55:06.191581  4441 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:55:06.218504  4441 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:55:06.219444  4441 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:55:06.219714  4441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:55:06.238757  4441 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.126:38025
I20261118 02:55:06.238776  4508 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.126:38025 every 8 connection(s)
I20261118 02:55:06.240468  4441 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
I20261118 02:55:06.246315  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 4441
I20261118 02:55:06.248207  4509 sys_catalog.cc:263] Verifying existing consensus state
I20261118 02:55:06.252027  4509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Bootstrap starting.
I20261118 02:55:06.270102  4509 log.cc:826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:06.312275  4513 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:06.312477  4513 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:06.313138  4513 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:06.321758  4138 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:06.339010  4509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=85 overwritten=0 applied=84 ignored=0} inserts{seen=56 ignored=0} mutations{seen=70 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20261118 02:55:06.339728  4509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Bootstrap complete.
I20261118 02:55:06.347797  4509 raft_consensus.cc:360] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:06.351295  4509 raft_consensus.cc:748] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Initialized, Role: FOLLOWER
I20261118 02:55:06.352030  4509 consensus_queue.cc:261] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 84, Last appended: 4.85, Last appended by leader: 85, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:06.352927  4518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:06.353240  4518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:06.354193  4509 sys_catalog.cc:565] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:55:06.355332  4484 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 5 candidate_status { last_received { term: 4 index: 93 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:55:06.355957  4484 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 813a6937cea442ef81645fccc32207f4 in term 4.
I20261118 02:55:06.356923  4139 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4; no voters: 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:06.357728  4513 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20261118 02:55:06.357846  4513 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:06.357932  4513 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 4 FOLLOWER]: Advancing to term 5
I20261118 02:55:06.361764  4513 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:06.366215  4513 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 election: Requested vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:06.366958  4138 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:06.368651  4484 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 5 candidate_status { last_received { term: 4 index: 93 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d"
I20261118 02:55:06.368886  4484 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 4 FOLLOWER]: Advancing to term 5
I20261118 02:55:06.370718  3800 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:55:06.375556  3663 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:55:06.381558  4484 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 813a6937cea442ef81645fccc32207f4 in term 5.
I20261118 02:55:06.382061  4139 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4; no voters: 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:06.382437  4513 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 5 FOLLOWER]: Leader election won for term 5
I20261118 02:55:06.383000  4513 raft_consensus.cc:705] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 5 LEADER]: Becoming Leader. State: Replica: 813a6937cea442ef81645fccc32207f4, State: Running, Role: LEADER
I20261118 02:55:06.383437  4513 consensus_queue.cc:238] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 93, Committed index: 93, Last appended: 4.93, Last appended by leader: 93, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:06.385298  4532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 813a6937cea442ef81645fccc32207f4. Latest consensus state: current_term: 5 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:06.385601  4532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:06.392979  4536 catalog_manager.cc:820] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20261118 02:55:06.396342  4138 consensus_peers.cc:597] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 -> Peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Couldn't send request to peer 457eb38cba564ecd9f6f11245172260b. Status: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:06.398357  4539 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:55:06.402362  4539 catalog_manager.cc:694] Loaded metadata for table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]
I20261118 02:55:06.402966  4539 catalog_manager.cc:694] Loaded metadata for table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]
I20261118 02:55:06.403434  4539 catalog_manager.cc:694] Loaded metadata for table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]
I20261118 02:55:06.403913  4539 catalog_manager.cc:694] Loaded metadata for table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]
I20261118 02:55:06.404709  4539 catalog_manager.cc:694] Loaded metadata for table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]
I20261118 02:55:06.405172  4539 catalog_manager.cc:694] Loaded metadata for table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]
I20261118 02:55:06.405908  4539 catalog_manager.cc:694] Loaded metadata for table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da]
I20261118 02:55:06.406381  4539 catalog_manager.cc:694] Loaded metadata for table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
I20261118 02:55:06.406839  4539 catalog_manager.cc:694] Loaded metadata for table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]
I20261118 02:55:06.407336  4539 catalog_manager.cc:694] Loaded metadata for table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:06.407795  4539 catalog_manager.cc:694] Loaded metadata for table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:06.408233  4539 catalog_manager.cc:694] Loaded metadata for table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b]
I20261118 02:55:06.408658  4539 catalog_manager.cc:694] Loaded metadata for table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]
I20261118 02:55:06.409092  4539 catalog_manager.cc:694] Loaded metadata for table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]
I20261118 02:55:06.409512  4539 catalog_manager.cc:694] Loaded metadata for table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]
I20261118 02:55:06.409970  4539 catalog_manager.cc:694] Loaded metadata for table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]
I20261118 02:55:06.410444  4539 catalog_manager.cc:694] Loaded metadata for table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:06.410789  4484 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Refusing update from remote peer 813a6937cea442ef81645fccc32207f4: Log matching property violated. Preceding OpId in replica: term: 4 index: 85. Preceding OpId from leader: term: 5 index: 94. (index mismatch)
I20261118 02:55:06.410887  4539 catalog_manager.cc:694] Loaded metadata for table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]
I20261118 02:55:06.411294  4539 catalog_manager.cc:694] Loaded metadata for table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]
I20261118 02:55:06.411914  4539 catalog_manager.cc:694] Loaded metadata for table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]
I20261118 02:55:06.411827  4541 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 94, Last known committed idx: 84, Time since last communication: 0.000s
I20261118 02:55:06.414538  4539 tablet_loader.cc:96] loaded metadata for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7])
I20261118 02:55:06.415033  4539 tablet_loader.cc:96] loaded metadata for tablet 0e901a03b90e4c978eb98be87163cfe5 (table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190])
I20261118 02:55:06.415480  4539 tablet_loader.cc:96] loaded metadata for tablet 1904140da78b4567a9decb250c6f74a3 (table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f])
I20261118 02:55:06.415928  4539 tablet_loader.cc:96] loaded metadata for tablet 1d347c0ae53241708ec623562f4740d4 (table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b])
I20261118 02:55:06.416352  4539 tablet_loader.cc:96] loaded metadata for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db])
I20261118 02:55:06.416790  4539 tablet_loader.cc:96] loaded metadata for tablet 2c92955aa65142f1bd20f4b8f3321500 (table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d])
I20261118 02:55:06.418406  4539 tablet_loader.cc:96] loaded metadata for tablet 5f05a8e5b21945a5bd3df86a5467a615 (table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf])
I20261118 02:55:06.418672  4518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 813a6937cea442ef81645fccc32207f4. Latest consensus state: current_term: 5 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:06.418823  4539 tablet_loader.cc:96] loaded metadata for tablet 65bd6fcaed5b47c3a2478d375ace9636 (table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73])
I20261118 02:55:06.418919  4518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:06.419395  4539 tablet_loader.cc:96] loaded metadata for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b])
I20261118 02:55:06.419799  4539 tablet_loader.cc:96] loaded metadata for tablet 78189cccd457466bb5d01231038fec24 (table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da])
I20261118 02:55:06.420197  4539 tablet_loader.cc:96] loaded metadata for tablet 8bed657d0da64d95851f8a86216b6c2c (table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c])
I20261118 02:55:06.420630  4539 tablet_loader.cc:96] loaded metadata for tablet 8f853e6b923444efa79cfbab2e31540e (table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec])
I20261118 02:55:06.421502  4539 tablet_loader.cc:96] loaded metadata for tablet a013d180f2cd430ab1ba30b39fd27591 (table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d])
I20261118 02:55:06.422183  4539 tablet_loader.cc:96] loaded metadata for tablet b129518ce23c45758ebcaa55410757d8 (table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1])
I20261118 02:55:06.423887  4518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 5 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:06.424127  4518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:06.428741  4539 tablet_loader.cc:96] loaded metadata for tablet b610080fc91745c7be260699946d5681 (table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139])
I20261118 02:55:06.429382  4539 tablet_loader.cc:96] loaded metadata for tablet c92c29c30ab6411dabe6f02331d29b7a (table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d])
I20261118 02:55:06.430320  4539 tablet_loader.cc:96] loaded metadata for tablet ccf78258c9634540a178855a142f4d4b (table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e])
I20261118 02:55:06.426038  4441 master_runner.cc:187] Error getting master registration for 127.2.229.125:38339: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111), 
I20261118 02:55:06.430853  4441 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:06.430922  4539 tablet_loader.cc:96] loaded metadata for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6])
I20261118 02:55:06.432561  4539 tablet_loader.cc:96] loaded metadata for tablet f094af4ac3604326a539461d06b01868 (table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67])
I20261118 02:55:06.433629  4539 tablet_loader.cc:96] loaded metadata for tablet ffe8d4c111d14054b76f9c7724072a94 (table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66])
I20261118 02:55:06.434425  4539 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:55:06.435967  4539 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:06.436036  4539 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:55:06.437469  4539 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:55:06.438617  4539 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Loaded TSK: 0
I20261118 02:55:06.439740  4539 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:55:06.439996  4513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 5 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:06.440294  4513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:06.440943  4539 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:55:06.441959  4539 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 20
I20261118 02:55:06.488780  4150 master_service.cc:965] ReplaceTablet: received request to replace tablet f094af4ac3604326a539461d06b01868 from {username='slave'} at 127.0.0.1:34238
I20261118 02:55:06.489176  4145 master_service.cc:965] ReplaceTablet: received request to replace tablet 8f853e6b923444efa79cfbab2e31540e from {username='slave'} at 127.0.0.1:34238
I20261118 02:55:06.487730  4149 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34238:
name: "default.Table_51d36ff687ca47b5995e5693d7b8931b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:06.487991  4152 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34238:
name: "default.Table_1d3bb660e14d4101996960be8caa8f81"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:06.487829  4146 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34238:
table { table_name: "default.Table_287099a68a3540f1be0284c71d4d7056" } modify_external_catalogs: true
I20261118 02:55:06.488462  4148 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34238:
name: "default.Table_ee3815d9524243ec886bd171ccaff9e2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:06.493001  4146 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34238:
table { table_name: "default.Table_287099a68a3540f1be0284c71d4d7056" } modify_external_catalogs: true
I20261118 02:55:06.490669  4153 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34238:
name: "default.Table_52fca7a2e65f4e679bbef9de3df569f1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:06.490001  4151 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34238:
name: "default.Table_8139578347f041afbe51671d3be78447"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:06.488179  4147 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34238:
table { table_name: "default.Table_571d1360811c42dc815efb439b2b52e6" } modify_external_catalogs: true
I20261118 02:55:06.488986  4144 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34238:
name: "default.Table_40868189cecf45d0b705461c4328ee17"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:06.493492  4147 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34238:
table { table_name: "default.Table_571d1360811c42dc815efb439b2b52e6" } modify_external_catalogs: true
F20261118 02:55:06.502859  4205 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 15 received events: NotificationEvent(eventId=21, eventTime=1794970503, eventType=CREATE_TABLE, dbName=default, tableName=table_f0ae490a0df64e1097e144c7a370df61, message={"server":"","servicePrincipal":"","db":"default","table":"table_f0ae490a0df64e1097e144c7a370df61","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f0ae490a0df64e1097e144c7a370df61\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970503},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_f0ae490a0df64e1097e144c7a370df61\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f0ae490a0df64e1097e144c7a370df61\",\"kudu.table_id\":\"a59b9425c01049269883cf7767936c67\",\"transient_lastDdlTime\":\"1794970503\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1794970503, eventType=DROP_TABLE, dbName=default, tableName=table_c113c2b57e17479e92e241234a241af7, message={"server":"","servicePrincipal":"","db":"default","table":"table_c113c2b57e17479e92e241234a241af7","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c113c2b57e17479e92e241234a241af7\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970500},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_c113c2b57e17479e92e241234a241af7\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c113c2b57e17479e92e241234a241af7\",\"kudu.table_id\":\"da0cb25ccfed426ca61f0dd9f93d30ec\",\"transient_lastDdlTime\":\"1794970500\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
    @     0x7ff7582a7d5d  google::LogMessage::Fail() at ??:0
    @     0x7ff7582abb23  google::LogMessage::SendToLog() at ??:0
    @     0x7ff7582a775c  google::LogMessage::Flush() at ??:0
    @     0x7ff7582a79a1  google::LogMessage::~LogMessage() at ??:0
I20261118 02:55:06.517438  4145 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Sending DeleteTablet for 3 replicas of tablet 8f853e6b923444efa79cfbab2e31540e
I20261118 02:55:06.524168  4150 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Sending DeleteTablet for 3 replicas of tablet f094af4ac3604326a539461d06b01868
I20261118 02:55:06.524173  4145 catalog_manager.cc:6936] ReplaceTablet: tablet 8f853e6b923444efa79cfbab2e31540e deleted and replaced by tablet c0713dcf79f54873a99d7556ad8f1454
I20261118 02:55:06.529067  4145 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34238:
name: "default.Table_bdbed231860946c2bf9b514f724bc3e9"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:06.530020  4150 catalog_manager.cc:6936] ReplaceTablet: tablet f094af4ac3604326a539461d06b01868 deleted and replaced by tablet ab61b43e639a4e75bc2e7009ea86a64f
I20261118 02:55:06.530406  4150 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34238:
table { table_name: "default.Table_87d592a09c3348028719a603956fbce6" } modify_external_catalogs: true
I20261118 02:55:06.530674  4150 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34238:
table { table_name: "default.Table_87d592a09c3348028719a603956fbce6" } modify_external_catalogs: true
    @     0x7ff76d7279a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 02:55:06.538004  3595 tablet_service.cc:1547] Processing DeleteTablet for tablet 8f853e6b923444efa79cfbab2e31540e with delete_type TABLET_DATA_DELETED (replaced by tablet c0713dcf79f54873a99d7556ad8f1454) from {username='slave'} at 127.0.0.1:33682
I20261118 02:55:06.538363  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet f094af4ac3604326a539461d06b01868 with delete_type TABLET_DATA_DELETED (replaced by tablet ab61b43e639a4e75bc2e7009ea86a64f) from {username='slave'} at 127.0.0.1:33682
I20261118 02:55:06.554519  4558 tablet_replica.cc:333] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:06.554770  4558 raft_consensus.cc:2261] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:06.555022  4558 raft_consensus.cc:2290] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:06.555778  4558 ts_tablet_manager.cc:1962] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:06.559333  3595 tablet_service.cc:1500] Processing CreateTablet for tablet c0713dcf79f54873a99d7556ad8f1454 (DEFAULT_TABLE table=default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:06.559973  3595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0713dcf79f54873a99d7556ad8f1454. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:06.560336  3597 tablet_service.cc:1500] Processing CreateTablet for tablet ab61b43e639a4e75bc2e7009ea86a64f (DEFAULT_TABLE table=default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:06.560724  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab61b43e639a4e75bc2e7009ea86a64f. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7ff76d726251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
    @     0x7ff76d72decc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
    @     0x7ff76d72dd49  std::_Function_handler<>::_M_invoke() at ??:0
I20261118 02:55:06.576813  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet f094af4ac3604326a539461d06b01868 with delete_type TABLET_DATA_DELETED (replaced by tablet ab61b43e639a4e75bc2e7009ea86a64f) from {username='slave'} at 127.0.0.1:56632
I20261118 02:55:06.576848  3734 tablet_service.cc:1547] Processing DeleteTablet for tablet 8f853e6b923444efa79cfbab2e31540e with delete_type TABLET_DATA_DELETED (replaced by tablet c0713dcf79f54873a99d7556ad8f1454) from {username='slave'} at 127.0.0.1:48148
I20261118 02:55:06.577010  3880 tablet_service.cc:1547] Processing DeleteTablet for tablet 8f853e6b923444efa79cfbab2e31540e with delete_type TABLET_DATA_DELETED (replaced by tablet c0713dcf79f54873a99d7556ad8f1454) from {username='slave'} at 127.0.0.1:56632
I20261118 02:55:06.577426  4558 ts_tablet_manager.cc:1975] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:06.577543  4558 log.cc:1199] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/8f853e6b923444efa79cfbab2e31540e
I20261118 02:55:06.577507  3879 tablet_service.cc:1500] Processing CreateTablet for tablet c0713dcf79f54873a99d7556ad8f1454 (DEFAULT_TABLE table=default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:06.577483  3882 tablet_service.cc:1500] Processing CreateTablet for tablet ab61b43e639a4e75bc2e7009ea86a64f (DEFAULT_TABLE table=default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:06.577947  3879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0713dcf79f54873a99d7556ad8f1454. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:06.577948  4558 ts_tablet_manager.cc:1996] T 8f853e6b923444efa79cfbab2e31540e P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:06.578984  4558 tablet_replica.cc:333] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:06.579005  4140 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet 8f853e6b923444efa79cfbab2e31540e (table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]) successfully deleted
I20261118 02:55:06.579177  4558 raft_consensus.cc:2261] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:06.579331  4558 raft_consensus.cc:2290] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:06.579869  4558 ts_tablet_manager.cc:1962] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:06.581113  3735 tablet_service.cc:1547] Processing DeleteTablet for tablet f094af4ac3604326a539461d06b01868 with delete_type TABLET_DATA_DELETED (replaced by tablet ab61b43e639a4e75bc2e7009ea86a64f) from {username='slave'} at 127.0.0.1:48148
I20261118 02:55:06.581939  3735 tablet_service.cc:1500] Processing CreateTablet for tablet c0713dcf79f54873a99d7556ad8f1454 (DEFAULT_TABLE table=default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:06.582327  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0713dcf79f54873a99d7556ad8f1454. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:06.584251  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab61b43e639a4e75bc2e7009ea86a64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:06.585860  4558 ts_tablet_manager.cc:1975] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:06.585973  4558 log.cc:1199] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/f094af4ac3604326a539461d06b01868
I20261118 02:55:06.586333  4558 ts_tablet_manager.cc:1996] T f094af4ac3604326a539461d06b01868 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:06.587374  4140 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet f094af4ac3604326a539461d06b01868 (table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]) successfully deleted
I20261118 02:55:06.587672  4563 tablet_bootstrap.cc:492] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:06.589735  4563 tablet_bootstrap.cc:654] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:06.590438  3733 tablet_service.cc:1500] Processing CreateTablet for tablet ab61b43e639a4e75bc2e7009ea86a64f (DEFAULT_TABLE table=default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:06.590842  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab61b43e639a4e75bc2e7009ea86a64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:06.591141  4559 tablet_bootstrap.cc:492] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:06.592049  4563 tablet_bootstrap.cc:492] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:06.592204  4563 ts_tablet_manager.cc:1449] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.005s	user 0.001s	sys 0.003s
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 02:55:06.592983  4559 tablet_bootstrap.cc:654] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:06.592933  4563 raft_consensus.cc:360] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.593106  4563 raft_consensus.cc:386] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:06.593182  4563 raft_consensus.cc:748] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:06.593428  4563 consensus_queue.cc:261] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.603297  3947 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:55:06.603652  4563 ts_tablet_manager.cc:1480] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.000s
I20261118 02:55:06.603997  4563 tablet_bootstrap.cc:492] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
    @     0x7ff7596f0dc9  kudu::Thread::SuperviseThread() at ??:0
I20261118 02:55:06.606629  4474 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" instance_seqno: 1794970499527172) as {username='slave'} at 127.2.229.67:39447; Asking this server to re-register.
    @     0x7ff75cdcb6db  start_thread at ??:0
I20261118 02:55:06.607681  3947 heartbeater.cc:462] Registering TS with master...
    @     0x7ff75583b61f  clone at ??:0
I20261118 02:55:06.609266  4564 tablet_bootstrap.cc:492] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:06.610337  4474 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:06.611084  4559 tablet_bootstrap.cc:492] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:06.611208  4564 tablet_bootstrap.cc:654] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:06.611228  4559 ts_tablet_manager.cc:1449] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.000s
I20261118 02:55:06.611785  4563 tablet_bootstrap.cc:654] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:06.613248  4559 raft_consensus.cc:360] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.613437  4559 raft_consensus.cc:386] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:06.613516  4559 raft_consensus.cc:748] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:06.614686  4559 consensus_queue.cc:261] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.615836  4561 tablet_replica.cc:333] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:06.616035  4561 raft_consensus.cc:2261] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:06.616442  4561 raft_consensus.cc:2290] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:06.617007  4561 ts_tablet_manager.cc:1962] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:06.618352  4559 ts_tablet_manager.cc:1480] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.000s
I20261118 02:55:06.619112  4474 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" instance_seqno: 1794970498532688) as {username='slave'} at 127.2.229.65:52805; Asking this server to re-register.
I20261118 02:55:06.620012  3663 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:06.622020  4474 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:06.622129  4559 tablet_bootstrap.cc:492] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:06.623769  4563 tablet_bootstrap.cc:492] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:06.623953  4563 ts_tablet_manager.cc:1449] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.001s
I20261118 02:55:06.624855  4563 raft_consensus.cc:360] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.625077  4563 raft_consensus.cc:386] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:06.625144  4563 raft_consensus.cc:748] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:06.625272  4562 tablet_replica.cc:333] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:06.625474  4562 raft_consensus.cc:2261] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:06.625420  4563 consensus_queue.cc:261] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.625664  4562 raft_consensus.cc:2290] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:06.626094  4563 ts_tablet_manager.cc:1480] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.002s
I20261118 02:55:06.626240  4562 ts_tablet_manager.cc:1962] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:06.626706  4559 tablet_bootstrap.cc:654] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:06.632683  4564 tablet_bootstrap.cc:492] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:06.632843  4564 ts_tablet_manager.cc:1449] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.024s	user 0.003s	sys 0.000s
I20261118 02:55:06.633646  4564 raft_consensus.cc:360] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.633908  4564 raft_consensus.cc:386] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:06.634002  4564 raft_consensus.cc:748] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:06.634089  4562 ts_tablet_manager.cc:1975] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:06.634209  4562 log.cc:1199] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/8f853e6b923444efa79cfbab2e31540e
I20261118 02:55:06.634241  4564 consensus_queue.cc:261] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.634860  4564 ts_tablet_manager.cc:1480] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.000s
I20261118 02:55:06.640374  4561 ts_tablet_manager.cc:1975] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:06.640482  4561 log.cc:1199] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/f094af4ac3604326a539461d06b01868
I20261118 02:55:06.640900  4562 ts_tablet_manager.cc:1996] T 8f853e6b923444efa79cfbab2e31540e P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:06.640919  4561 ts_tablet_manager.cc:1996] T f094af4ac3604326a539461d06b01868 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:06.642283  4561 tablet_replica.cc:333] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:06.642499  4561 raft_consensus.cc:2261] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:06.642822  4561 raft_consensus.cc:2290] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:06.643400  4561 ts_tablet_manager.cc:1962] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:06.643483  4474 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" instance_seqno: 1794970499028671) as {username='slave'} at 127.2.229.66:40409; Asking this server to re-register.
I20261118 02:55:06.644404  3800 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:06.646291  4559 tablet_bootstrap.cc:492] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:06.646449  4559 ts_tablet_manager.cc:1449] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.024s	user 0.001s	sys 0.004s
I20261118 02:55:06.646463  4474 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:06.647239  4559 raft_consensus.cc:360] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.647430  4559 raft_consensus.cc:386] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:06.647502  4559 raft_consensus.cc:748] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:06.647732  4559 consensus_queue.cc:261] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.648499  4559 ts_tablet_manager.cc:1480] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.002s
I20261118 02:55:06.648844  4564 tablet_bootstrap.cc:492] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:06.650580  4564 tablet_bootstrap.cc:654] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:06.651582  4562 tablet_replica.cc:333] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:06.651782  4562 raft_consensus.cc:2261] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:06.651947  4562 raft_consensus.cc:2290] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:06.652035  4561 ts_tablet_manager.cc:1975] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:06.652185  4561 log.cc:1199] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/8f853e6b923444efa79cfbab2e31540e
I20261118 02:55:06.652527  4561 ts_tablet_manager.cc:1996] T 8f853e6b923444efa79cfbab2e31540e P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:06.652566  4562 ts_tablet_manager.cc:1962] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:06.654943  4564 tablet_bootstrap.cc:492] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:06.655103  4564 ts_tablet_manager.cc:1449] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.006s	user 0.003s	sys 0.000s
I20261118 02:55:06.655970  4564 raft_consensus.cc:360] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.656200  4564 raft_consensus.cc:386] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:06.656279  4564 raft_consensus.cc:748] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:06.656518  4564 consensus_queue.cc:261] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.657032  4564 ts_tablet_manager.cc:1480] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:06.661882  4562 ts_tablet_manager.cc:1975] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:06.661980  4562 log.cc:1199] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/f094af4ac3604326a539461d06b01868
I20261118 02:55:06.662240  4562 ts_tablet_manager.cc:1996] T f094af4ac3604326a539461d06b01868 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
W20261118 02:55:06.801687  3837 connection.cc:573] server connection from 127.0.0.1:56632 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:06.801677  3689 connection.cc:573] server connection from 127.0.0.1:48148 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:06.801824  3956 connection.cc:573] client connection to 127.2.229.124:36695 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:06.802488  3956 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.229.124:36695)
W20261118 02:55:06.802471  3690 connection.cc:573] client connection to 127.2.229.124:36695 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:06.802870  3956 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.2.229.124:36695)
W20261118 02:55:06.802850  3553 connection.cc:573] client connection to 127.2.229.124:36695 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:06.803037  3956 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.2.229.124:36695)
W20261118 02:55:06.803194  3956 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.229.124:36695)
I20261118 02:55:06.925024  4384 raft_consensus.cc:499] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:06.925217  4384 raft_consensus.cc:521] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.925881  4384 leader_election.cc:290] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:06.926492  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c0713dcf79f54873a99d7556ad8f1454" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:06.926473  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c0713dcf79f54873a99d7556ad8f1454" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:06.926703  3751 raft_consensus.cc:2490] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:06.926741  3901 raft_consensus.cc:2490] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:06.927234  3553 leader_election.cc:304] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:06.927500  4384 raft_consensus.cc:2826] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:06.927603  4384 raft_consensus.cc:499] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:06.927668  4384 raft_consensus.cc:3096] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:06.929574  4384 raft_consensus.cc:521] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.930083  4384 leader_election.cc:290] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:06.930624  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c0713dcf79f54873a99d7556ad8f1454" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:06.930621  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c0713dcf79f54873a99d7556ad8f1454" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:06.930785  3751 raft_consensus.cc:3096] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:06.931218  3901 raft_consensus.cc:3096] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:06.933236  3751 raft_consensus.cc:2490] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:06.933617  3552 leader_election.cc:304] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:06.933676  3901 raft_consensus.cc:2490] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:06.933874  4384 raft_consensus.cc:2826] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:06.934281  4384 raft_consensus.cc:705] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:06.934760  4384 consensus_queue.cc:238] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:06.937630  4583 consensus_queue.cc:1049] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:06.938017  4583 consensus_queue.cc:1049] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:06.974995  4591 ts_tablet_manager.cc:979] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: Initiating tablet copy from peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:06.976112  4591 tablet_copy_client.cc:323] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: tablet copy: Beginning tablet copy session from remote peer at address 127.2.229.66:40377
I20261118 02:55:06.977246  3775 tablet_copy_service.cc:140] P 99a39b1d9d814deeb3a8a644e9b392ef: Received BeginTabletCopySession request for tablet ccf78258c9634540a178855a142f4d4b from peer 3f1973986ec1466eab5bec01dd81654f ({username='slave'} at 127.2.229.67:42005)
I20261118 02:55:06.977474  3775 tablet_copy_service.cc:161] P 99a39b1d9d814deeb3a8a644e9b392ef: Beginning new tablet copy session on tablet ccf78258c9634540a178855a142f4d4b from peer 3f1973986ec1466eab5bec01dd81654f at {username='slave'} at 127.2.229.67:42005: session id = 3f1973986ec1466eab5bec01dd81654f-ccf78258c9634540a178855a142f4d4b
I20261118 02:55:06.980862  3775 tablet_copy_source_session.cc:215] T ccf78258c9634540a178855a142f4d4b P 99a39b1d9d814deeb3a8a644e9b392ef: Tablet Copy: opened 0 blocks and 1 log segments
I20261118 02:55:06.982932  4591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf78258c9634540a178855a142f4d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:06.987776  4591 tablet_copy_client.cc:806] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: tablet copy: Starting download of 0 data blocks...
I20261118 02:55:06.988040  4591 tablet_copy_client.cc:670] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: tablet copy: Starting download of 1 WAL segments...
I20261118 02:55:06.990769  4591 tablet_copy_client.cc:538] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 02:55:06.993625  4591 tablet_bootstrap.cc:492] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:06.999177  4591 tablet_bootstrap.cc:492] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:06.999537  4591 tablet_bootstrap.cc:492] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: Bootstrap complete.
I20261118 02:55:06.999789  4591 ts_tablet_manager.cc:1449] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.006s	user 0.005s	sys 0.001s
I20261118 02:55:07.001077  4591 raft_consensus.cc:360] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:07.001253  4591 raft_consensus.cc:748] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:07.001508  4591 consensus_queue.cc:261] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:07.002025  4591 ts_tablet_manager.cc:1480] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:07.003533  3775 tablet_copy_service.cc:342] P 99a39b1d9d814deeb3a8a644e9b392ef: Request end of tablet copy session 3f1973986ec1466eab5bec01dd81654f-ccf78258c9634540a178855a142f4d4b received from {username='slave'} at 127.2.229.67:42005
I20261118 02:55:07.003810  3775 tablet_copy_service.cc:434] P 99a39b1d9d814deeb3a8a644e9b392ef: ending tablet copy session 3f1973986ec1466eab5bec01dd81654f-ccf78258c9634540a178855a142f4d4b on tablet ccf78258c9634540a178855a142f4d4b with peer 3f1973986ec1466eab5bec01dd81654f
I20261118 02:55:07.062830  4594 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:07.063031  4594 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:07.064950  4594 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:07.065847  4459 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:07.067314  4460 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
W20261118 02:55:07.067639  4459 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:07.067819  4459 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d; no voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:07.068499  4594 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 02:55:07.068884  4580 raft_consensus.cc:499] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:07.069039  4580 raft_consensus.cc:521] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:07.069810  4580 leader_election.cc:290] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:07.070294  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ab61b43e639a4e75bc2e7009ea86a64f" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:07.070519  3901 raft_consensus.cc:2490] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:07.070547  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ab61b43e639a4e75bc2e7009ea86a64f" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:07.070770  3616 raft_consensus.cc:2490] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:07.070971  3690 leader_election.cc:304] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:07.071256  4580 raft_consensus.cc:2826] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:07.071375  4580 raft_consensus.cc:499] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:07.071467  4580 raft_consensus.cc:3096] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:07.073422  4580 raft_consensus.cc:521] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:07.073949  4580 leader_election.cc:290] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:07.074515  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ab61b43e639a4e75bc2e7009ea86a64f" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:07.074532  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "ab61b43e639a4e75bc2e7009ea86a64f" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:07.074702  3901 raft_consensus.cc:3096] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:07.074720  3616 raft_consensus.cc:3096] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:07.077730  3616 raft_consensus.cc:2490] T ab61b43e639a4e75bc2e7009ea86a64f P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:07.077744  3901 raft_consensus.cc:2490] T ab61b43e639a4e75bc2e7009ea86a64f P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:07.078269  3690 leader_election.cc:304] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:07.078702  4580 raft_consensus.cc:2826] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:07.078831  4580 raft_consensus.cc:705] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:07.079126  4580 consensus_queue.cc:238] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:07.082173  4580 consensus_queue.cc:1049] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:07.082392  4379 consensus_queue.cc:1049] T ab61b43e639a4e75bc2e7009ea86a64f P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:07.397850  4536 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:07.397953  4536 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: loading cluster ID for follower catalog manager: success
I20261118 02:55:07.401039  4536 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: acquiring CA information for follower catalog manager: success
I20261118 02:55:07.403229  4536 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:55:07.798398  4608 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:07.798589  4608 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:07.799456  4608 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:07.801621  4459 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:07.801791  4460 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:07.801931  4460 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d; no voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:07.802199  4608 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 02:55:07.811005  3901 raft_consensus.cc:1227] T ccf78258c9634540a178855a142f4d4b P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1]   Dedup: 1.1->[]
I20261118 02:55:08.838246  4620 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:08.838436  4620 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:08.839131  4620 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:08.841564  4460 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
W20261118 02:55:08.841743  4459 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:08.841873  4459 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d; no voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:08.842139  4620 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 02:55:09.041154  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.125:38339
--webserver_interface=127.2.229.125
--webserver_port=39093
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 02:55:09.389125  4625 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:55:09.389431  4625 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:55:09.389555  4625 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:55:09.398095  4625 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:55:09.398192  4625 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:55:09.398248  4625 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:55:09.398312  4625 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:55:09.408986  4625 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.125:38339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.229.125
--webserver_port=39093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:55:09.410651  4625 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:55:09.412458  4625 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:55:09.420225  4636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:09.420219  4635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:09.420872  4638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:09.421380  4625 server_base.cc:1061] running on GCE node
I20261118 02:55:09.422025  4625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:55:09.423336  4625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:55:09.424528  4625 hybrid_clock.cc:648] HybridClock initialized: now 1794970509424478 us; error 45 us; skew 500 ppm
I20261118 02:55:09.424825  4625 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:55:09.427099  4625 webserver.cc:577] Webserver started at http://127.2.229.125:39093/ using document root <none> and password file <none>
I20261118 02:55:09.427888  4625 fs_manager.cc:362] Metadata directory not provided
I20261118 02:55:09.428033  4625 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:55:09.428408  4643 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:09.428597  4643 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:09.429253  4643 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:09.431502  4459 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:09.431672  4460 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:09.431792  4460 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d; no voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:09.432049  4643 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 02:55:09.436148  4625 fs_manager.cc:714] Time spent opening directory manager: real 0.006s	user 0.002s	sys 0.003s
I20261118 02:55:09.439002  4648 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:55:09.440493  4625 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20261118 02:55:09.440631  4625 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
uuid: "457eb38cba564ecd9f6f11245172260b"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:55:09.441294  4625 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:55:09.501787  4625 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:55:09.502708  4625 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:55:09.503031  4625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:55:09.523041  4625 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.125:38339
I20261118 02:55:09.523105  4699 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.125:38339 every 8 connection(s)
I20261118 02:55:09.524816  4625 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
I20261118 02:55:09.531870  4700 sys_catalog.cc:263] Verifying existing consensus state
I20261118 02:55:09.531965  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 4625
I20261118 02:55:09.536155  4700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap starting.
I20261118 02:55:09.558239  4700 log.cc:826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:09.610821  3665 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:09.632992  4700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap replayed 1/1 log segments. Stats: ops{read=93 overwritten=0 applied=93 ignored=0} inserts{seen=63 ignored=0} mutations{seen=79 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:09.633548  4700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap complete.
I20261118 02:55:09.638876  4700 raft_consensus.cc:360] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:09.639570  4700 raft_consensus.cc:748] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Initialized, Role: FOLLOWER
I20261118 02:55:09.640215  4700 consensus_queue.cc:261] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 93, Last appended: 4.93, Last appended by leader: 93, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:09.640946  4706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:09.641184  4706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:09.641933  4700 sys_catalog.cc:565] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:55:09.661449  4720 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:09.661569  4720 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: loading cluster ID for follower catalog manager: success
I20261118 02:55:09.667950  4720 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: acquiring CA information for follower catalog manager: success
I20261118 02:55:09.670645  4720 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:55:09.685621  4625 master_runner.cc:187] Error getting master registration for 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111), 
I20261118 02:55:09.691267  4625 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:09.836591  3949 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:09.838797  3802 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:09.839545  4665 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" instance_seqno: 1794970499527172) as {username='slave'} at 127.2.229.67:60229; Asking this server to re-register.
I20261118 02:55:09.840391  3949 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:09.841492  4665 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" instance_seqno: 1794970499028671) as {username='slave'} at 127.2.229.66:55747; Asking this server to re-register.
I20261118 02:55:09.842839  4665 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:09.843436  3802 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:09.845438  4664 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:10.078824  4706 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:10.079095  4706 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:10.081084  4706 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:10.081784  4484 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 5 candidate_status { last_received { term: 4 index: 93 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
W20261118 02:55:10.082010  4651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:10.082119  4484 raft_consensus.cc:2415] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 457eb38cba564ecd9f6f11245172260b in current term 5: Already voted for candidate 813a6937cea442ef81645fccc32207f4 in this term.
W20261118 02:55:10.083340  4651 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:10.083487  4651 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:10.083843  4706 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 4 FOLLOWER]: Advancing to term 5
I20261118 02:55:10.087531  4706 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20261118 02:55:10.116843  3665 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:10.119119  4664 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" instance_seqno: 1794970498532688) as {username='slave'} at 127.2.229.65:45613; Asking this server to re-register.
I20261118 02:55:10.119891  3665 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:10.121568  4664 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:10.608711  4730 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:10.608884  4730 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:10.609583  4730 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:10.610154  4484 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 6 candidate_status { last_received { term: 4 index: 93 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:55:10.610455  4484 raft_consensus.cc:2432] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 457eb38cba564ecd9f6f11245172260b for term 6 because replica has last-logged OpId of term: 5 index: 97, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 93.
W20261118 02:55:10.612221  4651 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:10.612418  4651 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:10.613039  4730 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 02:55:11.380443  4738 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:11.380635  4738 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:11.381322  4738 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:11.381956  4484 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 6 candidate_status { last_received { term: 4 index: 93 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:55:11.382246  4484 raft_consensus.cc:2432] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 457eb38cba564ecd9f6f11245172260b for term 6 because replica has last-logged OpId of term: 5 index: 97, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 93.
W20261118 02:55:11.383311  4651 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:11.383459  4651 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:11.383692  4738 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20261118 02:55:11.442483  4441 master_runner.cc:187] Error getting master registration for 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111), 
I20261118 02:55:11.442680  4441 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:11.547122  4743 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:11.547319  4743 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:11.548496  4743 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:11.548717  4675 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 6 candidate_status { last_received { term: 5 index: 97 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b" is_pre_election: true
I20261118 02:55:11.549162  4675 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 5.
I20261118 02:55:11.549705  4459 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b; no voters: 
W20261118 02:55:11.549947  4460 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:11.549986  4743 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20261118 02:55:11.550318  4743 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:11.550421  4743 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 5 FOLLOWER]: Advancing to term 6
I20261118 02:55:11.553462  4743 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:11.554422  4743 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 election: Requested vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:11.554540  4675 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 6 candidate_status { last_received { term: 5 index: 97 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b"
I20261118 02:55:11.554720  4675 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 5 FOLLOWER]: Advancing to term 6
W20261118 02:55:11.555732  4460 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:11.557613  4675 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 6.
I20261118 02:55:11.558022  4459 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b; no voters: 813a6937cea442ef81645fccc32207f4
I20261118 02:55:11.558318  4743 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 6 FOLLOWER]: Leader election won for term 6
I20261118 02:55:11.558694  4743 raft_consensus.cc:705] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 6 LEADER]: Becoming Leader. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Running, Role: LEADER
I20261118 02:55:11.559206  4743 consensus_queue.cc:238] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 97, Committed index: 97, Last appended: 5.97, Last appended by leader: 97, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:11.561024  4745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 6 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:11.561278  4745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:11.566282  4748 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
W20261118 02:55:11.566767  4460 consensus_peers.cc:597] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d -> Peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Couldn't send request to peer 813a6937cea442ef81645fccc32207f4. Status: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:11.569782  4675 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Refusing update from remote peer 26c6b20fa203404a88463ea27bdd3c7d: Log matching property violated. Preceding OpId in replica: term: 4 index: 93. Preceding OpId from leader: term: 6 index: 98. (index mismatch)
I20261118 02:55:11.569769  4748 catalog_manager.cc:694] Loaded metadata for table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]
I20261118 02:55:11.570315  4748 catalog_manager.cc:694] Loaded metadata for table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]
I20261118 02:55:11.570602  4747 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 98, Last known committed idx: 93, Time since last communication: 0.000s
I20261118 02:55:11.570757  4748 catalog_manager.cc:694] Loaded metadata for table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]
I20261118 02:55:11.571241  4748 catalog_manager.cc:694] Loaded metadata for table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]
I20261118 02:55:11.571622  4748 catalog_manager.cc:694] Loaded metadata for table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]
I20261118 02:55:11.572063  4748 catalog_manager.cc:694] Loaded metadata for table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]
I20261118 02:55:11.572690  4748 catalog_manager.cc:694] Loaded metadata for table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da]
I20261118 02:55:11.573125  4748 catalog_manager.cc:694] Loaded metadata for table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
I20261118 02:55:11.573521  4748 catalog_manager.cc:694] Loaded metadata for table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]
I20261118 02:55:11.573925  4748 catalog_manager.cc:694] Loaded metadata for table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:11.574307  4748 catalog_manager.cc:694] Loaded metadata for table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:11.574739  4748 catalog_manager.cc:694] Loaded metadata for table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b]
I20261118 02:55:11.575223  4748 catalog_manager.cc:694] Loaded metadata for table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]
I20261118 02:55:11.575758  4748 catalog_manager.cc:694] Loaded metadata for table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]
I20261118 02:55:11.576308  4748 catalog_manager.cc:694] Loaded metadata for table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]
I20261118 02:55:11.576771  4748 catalog_manager.cc:694] Loaded metadata for table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]
I20261118 02:55:11.577324  4748 catalog_manager.cc:694] Loaded metadata for table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:11.577795  4748 catalog_manager.cc:694] Loaded metadata for table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]
I20261118 02:55:11.578333  4748 catalog_manager.cc:694] Loaded metadata for table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]
I20261118 02:55:11.578850  4748 catalog_manager.cc:694] Loaded metadata for table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]
I20261118 02:55:11.578821  4738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 6 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:11.579099  4738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:11.579442  4751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:11.579715  4751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:11.580889  4749 mvcc.cc:204] Tried to move back new op lower bound from 7352199194737414144 to 7352199194150617088. Current Snapshot: MvccSnapshot[applied={T|T < 7352199194737414144 or (T in {7352199194737414144})}]
I20261118 02:55:11.581488  4748 tablet_loader.cc:96] loaded metadata for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7])
I20261118 02:55:11.581549  4743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:11.581795  4743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:11.581969  4748 tablet_loader.cc:96] loaded metadata for tablet 0e901a03b90e4c978eb98be87163cfe5 (table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190])
I20261118 02:55:11.582358  4748 tablet_loader.cc:96] loaded metadata for tablet 1904140da78b4567a9decb250c6f74a3 (table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f])
I20261118 02:55:11.582676  4748 tablet_loader.cc:96] loaded metadata for tablet 1d347c0ae53241708ec623562f4740d4 (table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b])
I20261118 02:55:11.582904  4751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:11.583074  4751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:11.583110  4748 tablet_loader.cc:96] loaded metadata for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db])
I20261118 02:55:11.583428  4748 tablet_loader.cc:96] loaded metadata for tablet 2c92955aa65142f1bd20f4b8f3321500 (table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d])
I20261118 02:55:11.584581  4748 tablet_loader.cc:96] loaded metadata for tablet 5f05a8e5b21945a5bd3df86a5467a615 (table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf])
I20261118 02:55:11.584870  4748 tablet_loader.cc:96] loaded metadata for tablet 65bd6fcaed5b47c3a2478d375ace9636 (table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73])
I20261118 02:55:11.585318  4748 tablet_loader.cc:96] loaded metadata for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b])
I20261118 02:55:11.585609  4748 tablet_loader.cc:96] loaded metadata for tablet 78189cccd457466bb5d01231038fec24 (table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da])
I20261118 02:55:11.585911  4748 tablet_loader.cc:96] loaded metadata for tablet 8bed657d0da64d95851f8a86216b6c2c (table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c])
I20261118 02:55:11.586593  4748 tablet_loader.cc:96] loaded metadata for tablet a013d180f2cd430ab1ba30b39fd27591 (table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d])
I20261118 02:55:11.586977  4748 tablet_loader.cc:96] loaded metadata for tablet ab61b43e639a4e75bc2e7009ea86a64f (table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67])
I20261118 02:55:11.587219  4748 tablet_loader.cc:96] loaded metadata for tablet b129518ce23c45758ebcaa55410757d8 (table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1])
I20261118 02:55:11.587445  4748 tablet_loader.cc:96] loaded metadata for tablet b610080fc91745c7be260699946d5681 (table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139])
I20261118 02:55:11.587688  4748 tablet_loader.cc:96] loaded metadata for tablet c0713dcf79f54873a99d7556ad8f1454 (table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec])
I20261118 02:55:11.587950  4748 tablet_loader.cc:96] loaded metadata for tablet c92c29c30ab6411dabe6f02331d29b7a (table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d])
I20261118 02:55:11.588243  4748 tablet_loader.cc:96] loaded metadata for tablet ccf78258c9634540a178855a142f4d4b (table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e])
I20261118 02:55:11.588639  4748 tablet_loader.cc:96] loaded metadata for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6])
I20261118 02:55:11.589241  4748 tablet_loader.cc:96] loaded metadata for tablet ffe8d4c111d14054b76f9c7724072a94 (table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66])
I20261118 02:55:11.589478  4748 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:55:11.590610  4748 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:11.590677  4748 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:55:11.592012  4748 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:55:11.592973  4748 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Loaded TSK: 0
I20261118 02:55:11.593789  4748 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:55:11.594781  4748 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:55:11.595840  4748 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 20
I20261118 02:55:11.667223  4472 master_service.cc:965] ReplaceTablet: received request to replace tablet b610080fc91745c7be260699946d5681 from {username='slave'} at 127.0.0.1:46014
I20261118 02:55:11.667469  4473 master_service.cc:965] ReplaceTablet: received request to replace tablet 5f05a8e5b21945a5bd3df86a5467a615 from {username='slave'} at 127.0.0.1:46014
I20261118 02:55:11.668659  4467 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46014:
table { table_name: "default.Table_a14cb6414555446286bf35bc5c6fb910" } new_table_name: "default.Table_324a2600fef947748ef03e27f7cb7e09" modify_external_catalogs: true
I20261118 02:55:11.668612  4468 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46014:
name: "default.Table_fd4fcac1f1d946fa898176f1d5e87e9b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:11.669991  4474 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46014:
table { table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" } modify_external_catalogs: true
I20261118 02:55:11.670266  4474 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46014:
table { table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" } modify_external_catalogs: true
I20261118 02:55:11.671231  4469 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46014:
table { table_name: "default.Table_28c62f411da74545aa42dad7325bc064" } modify_external_catalogs: true
I20261118 02:55:11.671770  4469 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46014:
table { table_name: "default.Table_28c62f411da74545aa42dad7325bc064" } modify_external_catalogs: true
I20261118 02:55:11.670744  4471 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46014:
table { table_name: "default.Table_4f7796d5d5d54def843b971e66d15ea6" } new_table_name: "default.Table_79eb11797e3f4bc2bffe161096300f7a" modify_external_catalogs: true
I20261118 02:55:11.671438  4465 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46014:
name: "default.Table_dc982811b980416484416db1f74c685d"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:11.674870  4466 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46014:
table { table_name: "default.Table_8ae376d4d4d64414b5c08dc3c2e58bc4" } new_table_name: "default.Table_5091d82298ca44bea5335d4989eb02d8" modify_external_catalogs: true
I20261118 02:55:11.673579  4470 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46014:
name: "default.Table_e39f59e2fdbb47fda4c10beba29ad8f1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
F20261118 02:55:11.677047  4535 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 15 received events: NotificationEvent(eventId=21, eventTime=1794970503, eventType=CREATE_TABLE, dbName=default, tableName=table_f0ae490a0df64e1097e144c7a370df61, message={"server":"","servicePrincipal":"","db":"default","table":"table_f0ae490a0df64e1097e144c7a370df61","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f0ae490a0df64e1097e144c7a370df61\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970503},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_f0ae490a0df64e1097e144c7a370df61\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f0ae490a0df64e1097e144c7a370df61\",\"kudu.table_id\":\"a59b9425c01049269883cf7767936c67\",\"transient_lastDdlTime\":\"1794970503\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1794970503, eventType=DROP_TABLE, dbName=default, tableName=table_c113c2b57e17479e92e241234a241af7, message={"server":"","servicePrincipal":"","db":"default","table":"table_c113c2b57e17479e92e241234a241af7","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c113c2b57e17479e92e241234a241af7\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970500},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_c113c2b57e17479e92e241234a241af7\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c113c2b57e17479e92e241234a241af7\",\"kudu.table_id\":\"da0cb25ccfed426ca61f0dd9f93d30ec\",\"transient_lastDdlTime\":\"1794970500\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
    @     0x7f436c3add5d  google::LogMessage::Fail() at ??:0
    @     0x7f436c3b1b23  google::LogMessage::SendToLog() at ??:0
    @     0x7f436c3ad75c  google::LogMessage::Flush() at ??:0
I20261118 02:55:11.686000  4473 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet 5f05a8e5b21945a5bd3df86a5467a615
    @     0x7f436c3ad9a1  google::LogMessage::~LogMessage() at ??:0
I20261118 02:55:11.688660  4473 catalog_manager.cc:6936] ReplaceTablet: tablet 5f05a8e5b21945a5bd3df86a5467a615 deleted and replaced by tablet 46f05790dbdc4b8185a7b0362f604cf8
I20261118 02:55:11.689280  4472 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet b610080fc91745c7be260699946d5681
I20261118 02:55:11.689844  4472 catalog_manager.cc:6936] ReplaceTablet: tablet b610080fc91745c7be260699946d5681 deleted and replaced by tablet 973831392ec24e0681c3d4fffd8bb199
    @     0x7f438182d9a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 02:55:11.710685  3595 tablet_service.cc:1547] Processing DeleteTablet for tablet b610080fc91745c7be260699946d5681 with delete_type TABLET_DATA_DELETED (replaced by tablet 973831392ec24e0681c3d4fffd8bb199) from {username='slave'} at 127.0.0.1:33686
I20261118 02:55:11.710644  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet 5f05a8e5b21945a5bd3df86a5467a615 with delete_type TABLET_DATA_DELETED (replaced by tablet 46f05790dbdc4b8185a7b0362f604cf8) from {username='slave'} at 127.0.0.1:33686
I20261118 02:55:11.714581  4473 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46014:
table { table_name: "default.Table_0385cef6593c419eb426657c8cbef2bf" } new_table_name: "default.Table_496441d99faa4e2b8c7cd98e4a5ce745" modify_external_catalogs: true
    @     0x7f438182c251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 02:55:11.719249  4472 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46014:
name: "default.Table_51d36ff687ca47b5995e5693d7b8931b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
    @     0x7f4381833ecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 02:55:11.720479  4766 tablet_replica.cc:333] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:11.720708  4766 raft_consensus.cc:2261] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:11.721908  4766 raft_consensus.cc:2290] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:11.722530  4766 ts_tablet_manager.cc:1962] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
    @     0x7f4381833d49  std::_Function_handler<>::_M_invoke() at ??:0
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 02:55:11.726418  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet 5f05a8e5b21945a5bd3df86a5467a615 with delete_type TABLET_DATA_DELETED (replaced by tablet 46f05790dbdc4b8185a7b0362f604cf8) from {username='slave'} at 127.0.0.1:56640
I20261118 02:55:11.726590  3879 tablet_service.cc:1547] Processing DeleteTablet for tablet b610080fc91745c7be260699946d5681 with delete_type TABLET_DATA_DELETED (replaced by tablet 973831392ec24e0681c3d4fffd8bb199) from {username='slave'} at 127.0.0.1:56640
I20261118 02:55:11.727041  4767 tablet_replica.cc:333] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:11.727229  4767 raft_consensus.cc:2261] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:11.727582  4767 raft_consensus.cc:2290] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:11.728190  4767 ts_tablet_manager.cc:1962] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:11.729379  3881 tablet_service.cc:1500] Processing CreateTablet for tablet 973831392ec24e0681c3d4fffd8bb199 (DEFAULT_TABLE table=default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:11.729784  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973831392ec24e0681c3d4fffd8bb199. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:11.730211  4766 ts_tablet_manager.cc:1975] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:11.730320  4766 log.cc:1199] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/b610080fc91745c7be260699946d5681
I20261118 02:55:11.730760  4766 ts_tablet_manager.cc:1996] T b610080fc91745c7be260699946d5681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:11.731588  3595 tablet_service.cc:1500] Processing CreateTablet for tablet 973831392ec24e0681c3d4fffd8bb199 (DEFAULT_TABLE table=default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:11.731954  3595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973831392ec24e0681c3d4fffd8bb199. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:11.732034  3879 tablet_service.cc:1500] Processing CreateTablet for tablet 46f05790dbdc4b8185a7b0362f604cf8 (DEFAULT_TABLE table=default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:11.732493  3879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46f05790dbdc4b8185a7b0362f604cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:11.733669  3597 tablet_service.cc:1500] Processing CreateTablet for tablet 46f05790dbdc4b8185a7b0362f604cf8 (DEFAULT_TABLE table=default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:11.734030  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46f05790dbdc4b8185a7b0362f604cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:11.734999  4767 ts_tablet_manager.cc:1975] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:11.735173  4767 log.cc:1199] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/5f05a8e5b21945a5bd3df86a5467a615
I20261118 02:55:11.735579  4767 ts_tablet_manager.cc:1996] T 5f05a8e5b21945a5bd3df86a5467a615 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:11.736608  4767 tablet_replica.cc:333] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:11.736804  4767 raft_consensus.cc:2261] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:11.736778  4460 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet 5f05a8e5b21945a5bd3df86a5467a615 (table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]) successfully deleted
I20261118 02:55:11.736963  4767 raft_consensus.cc:2290] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:11.737495  4767 ts_tablet_manager.cc:1962] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:11.738997  4461 catalog_manager.cc:5065] TS 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): tablet b610080fc91745c7be260699946d5681 (table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]) successfully deleted
I20261118 02:55:11.739292  4766 tablet_replica.cc:333] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:11.739506  4766 raft_consensus.cc:2261] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:11.739673  4766 raft_consensus.cc:2290] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:11.740213  4766 ts_tablet_manager.cc:1962] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:11.741859  3734 tablet_service.cc:1547] Processing DeleteTablet for tablet 5f05a8e5b21945a5bd3df86a5467a615 with delete_type TABLET_DATA_DELETED (replaced by tablet 46f05790dbdc4b8185a7b0362f604cf8) from {username='slave'} at 127.0.0.1:48156
I20261118 02:55:11.742293  3735 tablet_service.cc:1547] Processing DeleteTablet for tablet b610080fc91745c7be260699946d5681 with delete_type TABLET_DATA_DELETED (replaced by tablet 973831392ec24e0681c3d4fffd8bb199) from {username='slave'} at 127.0.0.1:48156
I20261118 02:55:11.742537  3732 tablet_service.cc:1500] Processing CreateTablet for tablet 46f05790dbdc4b8185a7b0362f604cf8 (DEFAULT_TABLE table=default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:11.742699  3733 tablet_service.cc:1500] Processing CreateTablet for tablet 973831392ec24e0681c3d4fffd8bb199 (DEFAULT_TABLE table=default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:11.742998  3732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46f05790dbdc4b8185a7b0362f604cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:11.743232  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973831392ec24e0681c3d4fffd8bb199. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:11.743693  4767 ts_tablet_manager.cc:1975] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:11.743825  4767 log.cc:1199] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/b610080fc91745c7be260699946d5681
I20261118 02:55:11.744275  4767 ts_tablet_manager.cc:1996] T b610080fc91745c7be260699946d5681 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:11.745342  4460 catalog_manager.cc:5065] TS 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): tablet b610080fc91745c7be260699946d5681 (table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]) successfully deleted
    @     0x7f436d7f6dc9  kudu::Thread::SuperviseThread() at ??:0
I20261118 02:55:11.753643  4772 tablet_bootstrap.cc:492] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
    @     0x7f4370ed16db  start_thread at ??:0
    @     0x7f436994161f  clone at ??:0
I20261118 02:55:11.756464  4772 tablet_bootstrap.cc:654] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:11.756669  4770 tablet_replica.cc:333] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:11.757047  4770 raft_consensus.cc:2261] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:11.757342  4770 raft_consensus.cc:2290] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:11.758003  4770 ts_tablet_manager.cc:1962] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:11.758157  4768 tablet_bootstrap.cc:492] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:11.760226  4768 tablet_bootstrap.cc:654] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:11.760560  4769 tablet_bootstrap.cc:492] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:11.762404  4769 tablet_bootstrap.cc:654] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:11.762938  4768 tablet_bootstrap.cc:492] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:11.763072  4768 ts_tablet_manager.cc:1449] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20261118 02:55:11.764050  4772 tablet_bootstrap.cc:492] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:11.764220  4772 ts_tablet_manager.cc:1449] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.011s	user 0.004s	sys 0.001s
I20261118 02:55:11.764508  4770 ts_tablet_manager.cc:1975] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:11.764614  4770 log.cc:1199] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/5f05a8e5b21945a5bd3df86a5467a615
I20261118 02:55:11.765028  4770 ts_tablet_manager.cc:1996] T 5f05a8e5b21945a5bd3df86a5467a615 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:11.765053  4772 raft_consensus.cc:360] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:11.765252  4772 raft_consensus.cc:386] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:11.765337  4772 raft_consensus.cc:748] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:11.765617  4772 consensus_queue.cc:261] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:11.765971  4770 tablet_replica.cc:333] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:11.767235  4772 ts_tablet_manager.cc:1480] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.003s	user 0.000s	sys 0.002s
I20261118 02:55:11.767527  4772 tablet_bootstrap.cc:492] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:11.769248  4772 tablet_bootstrap.cc:654] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:11.769541  4766 ts_tablet_manager.cc:1975] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:11.769680  4766 log.cc:1199] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/5f05a8e5b21945a5bd3df86a5467a615
I20261118 02:55:11.770109  4766 ts_tablet_manager.cc:1996] T 5f05a8e5b21945a5bd3df86a5467a615 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:11.771042  4768 raft_consensus.cc:360] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:11.771230  4768 raft_consensus.cc:386] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:11.771314  4768 raft_consensus.cc:748] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:11.771592  4768 consensus_queue.cc:261] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:11.772011  3663 heartbeater.cc:500] Master 127.2.229.126:38025 was elected leader, sending a full tablet report...
I20261118 02:55:11.772783  4770 raft_consensus.cc:2261] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:11.774117  4768 ts_tablet_manager.cc:1480] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.000s
I20261118 02:55:11.774374  4768 tablet_bootstrap.cc:492] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:11.775724  4769 tablet_bootstrap.cc:492] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:11.775887  4769 ts_tablet_manager.cc:1449] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.015s	user 0.003s	sys 0.000s
I20261118 02:55:11.777119  4769 raft_consensus.cc:360] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:11.777314  4769 raft_consensus.cc:386] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:11.777390  4769 raft_consensus.cc:748] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:11.777617  4769 consensus_queue.cc:261] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:11.778069  4768 tablet_bootstrap.cc:654] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:11.779294  4770 raft_consensus.cc:2290] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:11.781677  4770 ts_tablet_manager.cc:1962] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:11.782068  4769 ts_tablet_manager.cc:1480] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.002s
I20261118 02:55:11.784134  4769 tablet_bootstrap.cc:492] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:11.787536  4769 tablet_bootstrap.cc:654] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:11.790158  4770 ts_tablet_manager.cc:1975] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:11.790294  4770 log.cc:1199] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/b610080fc91745c7be260699946d5681
I20261118 02:55:11.790688  4772 tablet_bootstrap.cc:492] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:11.790719  4770 ts_tablet_manager.cc:1996] T b610080fc91745c7be260699946d5681 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:11.790863  4772 ts_tablet_manager.cc:1449] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.023s	user 0.004s	sys 0.005s
I20261118 02:55:11.791970  4772 raft_consensus.cc:360] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:11.792196  4772 raft_consensus.cc:386] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:11.792260  4772 raft_consensus.cc:748] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:11.792502  4772 consensus_queue.cc:261] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:11.793334  4772 ts_tablet_manager.cc:1480] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:11.795405  4768 tablet_bootstrap.cc:492] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:11.795593  4768 ts_tablet_manager.cc:1449] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.008s
I20261118 02:55:11.795606  4769 tablet_bootstrap.cc:492] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:11.795755  4769 ts_tablet_manager.cc:1449] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.012s	user 0.000s	sys 0.004s
I20261118 02:55:11.796408  4768 raft_consensus.cc:360] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:11.796617  4768 raft_consensus.cc:386] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:11.796782  4768 raft_consensus.cc:748] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:11.797115  4768 consensus_queue.cc:261] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:11.797746  2965 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-task8fD3U4/build/asan/bin/kudu pid 4625
I20261118 02:55:11.796482  4769 raft_consensus.cc:360] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:11.798854  4769 raft_consensus.cc:386] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:11.798943  4769 raft_consensus.cc:748] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:11.799081  4768 ts_tablet_manager.cc:1480] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:11.799238  4769 consensus_queue.cc:261] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:11.803267  4769 ts_tablet_manager.cc:1480] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.001s
I20261118 02:55:11.808547  4695 generic_service.cc:196] Checking for leaks (request via RPC)
I20261118 02:55:11.921391  2965 external_mini_cluster.cc:1664] Killing /tmp/dist-test-task8fD3U4/build/asan/bin/kudu with pid 4625
W20261118 02:55:11.958244  3837 connection.cc:573] server connection from 127.0.0.1:56640 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:11.958298  3689 connection.cc:573] server connection from 127.0.0.1:48156 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:11.958316  3551 connection.cc:573] server connection from 127.0.0.1:33686 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:11.958477  3956 connection.cc:573] client connection to 127.2.229.126:38025 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:11.958549  3837 connection.cc:573] client connection to 127.2.229.126:38025 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:11.958592  3690 connection.cc:573] client connection to 127.2.229.126:38025 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:11.958904  3553 connection.cc:573] client connection to 127.2.229.126:38025 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:11.959170  3956 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.229.126:38025)
W20261118 02:55:11.959489  3956 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.2.229.126:38025)
W20261118 02:55:11.959905  3956 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.2.229.126:38025)
W20261118 02:55:11.960073  3956 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.229.126:38025)
W20261118 02:55:11.961609  4756 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:11.959989 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.2.229.126:38025 (local address 127.2.229.65:58997)
1118 02:55:11.960252 (+   263us) negotiation.cc:107] Waiting for socket to connect
1118 02:55:11.960270 (+    18us) client_negotiation.cc:175] Beginning negotiation
1118 02:55:11.960398 (+   128us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 02:55:11.961565 (+  1167us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":164}
W20261118 02:55:11.962599  4736 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:11.961274 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.2.229.126:38025 (local address 127.2.229.66:60635)
1118 02:55:11.961579 (+   305us) negotiation.cc:107] Waiting for socket to connect
1118 02:55:11.961594 (+    15us) client_negotiation.cc:175] Beginning negotiation
1118 02:55:11.961695 (+   101us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 02:55:11.962556 (+   861us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":185}
W20261118 02:55:11.962687  4737 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:11.960595 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.2.229.126:38025 (local address 127.2.229.67:45299)
1118 02:55:11.960835 (+   240us) negotiation.cc:107] Waiting for socket to connect
1118 02:55:11.960852 (+    17us) client_negotiation.cc:175] Beginning negotiation
1118 02:55:11.960967 (+   115us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 02:55:11.962642 (+  1675us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":142}
I20261118 02:55:12.013914  4581 raft_consensus.cc:499] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:12.014122  4581 raft_consensus.cc:521] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:12.014883  4581 leader_election.cc:290] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:12.015492  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "46f05790dbdc4b8185a7b0362f604cf8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:12.015506  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "46f05790dbdc4b8185a7b0362f604cf8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:12.015697  3901 raft_consensus.cc:2490] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:12.015723  3751 raft_consensus.cc:2490] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:12.016202  3553 leader_election.cc:304] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:12.016482  4581 raft_consensus.cc:2826] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:12.016603  4581 raft_consensus.cc:499] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:12.016687  4581 raft_consensus.cc:3096] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:12.018610  4581 raft_consensus.cc:521] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:12.019210  4581 leader_election.cc:290] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:12.019716  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "46f05790dbdc4b8185a7b0362f604cf8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:12.019703  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "46f05790dbdc4b8185a7b0362f604cf8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:12.019913  3901 raft_consensus.cc:3096] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:12.019930  3751 raft_consensus.cc:3096] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:12.022164  3751 raft_consensus.cc:2490] T 46f05790dbdc4b8185a7b0362f604cf8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:12.022176  3901 raft_consensus.cc:2490] T 46f05790dbdc4b8185a7b0362f604cf8 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:12.022570  3552 leader_election.cc:304] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:12.022851  4581 raft_consensus.cc:2826] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:12.023233  4581 raft_consensus.cc:705] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:12.023773  4581 consensus_queue.cc:238] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:12.031153  4794 consensus_queue.cc:1049] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:12.032168  4794 consensus_queue.cc:1049] T 46f05790dbdc4b8185a7b0362f604cf8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:12.229302  4614 raft_consensus.cc:499] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:12.229525  4614 raft_consensus.cc:521] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:12.230232  4614 leader_election.cc:290] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:12.230854  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "973831392ec24e0681c3d4fffd8bb199" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:12.230854  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "973831392ec24e0681c3d4fffd8bb199" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:12.231078  3616 raft_consensus.cc:2490] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:12.231088  3751 raft_consensus.cc:2490] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:12.231495  3836 leader_election.cc:304] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:12.231762  4614 raft_consensus.cc:2826] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:12.231863  4614 raft_consensus.cc:499] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:12.231945  4614 raft_consensus.cc:3096] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:12.233860  4614 raft_consensus.cc:521] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:12.234371  4614 leader_election.cc:290] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:12.234848  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "973831392ec24e0681c3d4fffd8bb199" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:12.234838  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "973831392ec24e0681c3d4fffd8bb199" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:12.235011  3616 raft_consensus.cc:3096] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:12.235010  3751 raft_consensus.cc:3096] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:12.236936  3616 raft_consensus.cc:2490] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:12.237143  3751 raft_consensus.cc:2490] T 973831392ec24e0681c3d4fffd8bb199 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:12.237317  3838 leader_election.cc:304] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:12.237659  4614 raft_consensus.cc:2826] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:12.238065  4614 raft_consensus.cc:705] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:12.239562  4614 consensus_queue.cc:238] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:12.244458  4805 consensus_queue.cc:1049] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:12.247144  4805 consensus_queue.cc:1049] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:12.541921  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.125:38339
--webserver_interface=127.2.229.125
--webserver_port=39093
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20261118 02:55:12.885465  4810 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:55:12.885756  4810 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:55:12.885866  4810 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:55:12.894454  4810 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:55:12.894559  4810 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:55:12.894636  4810 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:55:12.894699  4810 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:55:12.904956  4810 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.125:38339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.229.125
--webserver_port=39093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:55:12.906653  4810 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:55:12.908629  4810 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:55:12.916683  4821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:12.916708  4820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:12.917578  4823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:12.918057  4810 server_base.cc:1061] running on GCE node
I20261118 02:55:12.919044  4810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:55:12.920642  4810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:55:12.921864  4810 hybrid_clock.cc:648] HybridClock initialized: now 1794970512921809 us; error 46 us; skew 500 ppm
I20261118 02:55:12.922224  4810 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:55:12.924669  4810 webserver.cc:577] Webserver started at http://127.2.229.125:39093/ using document root <none> and password file <none>
I20261118 02:55:12.925457  4810 fs_manager.cc:362] Metadata directory not provided
I20261118 02:55:12.925565  4810 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:55:12.931278  4810 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20261118 02:55:12.934192  4829 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:55:12.935770  4810 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:12.935956  4810 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
uuid: "457eb38cba564ecd9f6f11245172260b"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:55:12.936721  4810 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:55:12.957002  4810 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:55:12.957846  4810 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:55:12.958112  4810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:55:12.976824  4810 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.125:38339
I20261118 02:55:12.976848  4880 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.125:38339 every 8 connection(s)
I20261118 02:55:12.978459  4810 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
I20261118 02:55:12.981709  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 4810
I20261118 02:55:12.986006  4881 sys_catalog.cc:263] Verifying existing consensus state
I20261118 02:55:12.989521  4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap starting.
I20261118 02:55:13.007787  4881 log.cc:826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:13.090138  4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap replayed 1/1 log segments. Stats: ops{read=101 overwritten=0 applied=101 ignored=0} inserts{seen=67 ignored=0} mutations{seen=87 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:13.090667  4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap complete.
I20261118 02:55:13.096048  4881 raft_consensus.cc:360] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:13.096746  4881 raft_consensus.cc:748] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Initialized, Role: FOLLOWER
I20261118 02:55:13.097409  4881 consensus_queue.cc:261] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 101, Last appended: 6.101, Last appended by leader: 101, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:13.098317  4885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:13.098572  4885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:13.099340  4881 sys_catalog.cc:565] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:55:13.112406  4899 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:13.112562  4899 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: loading cluster ID for follower catalog manager: success
I20261118 02:55:13.107885  4810 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:13.113579  4810 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:13.117264  4899 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: acquiring CA information for follower catalog manager: success
I20261118 02:55:13.119794  4899 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:55:13.273941  3802 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:13.279958  4846 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" instance_seqno: 1794970499028671) as {username='slave'} at 127.2.229.66:45637; Asking this server to re-register.
I20261118 02:55:13.281105  3802 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:13.283844  4845 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:13.287583  3665 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:13.288973  3949 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:13.290493  4845 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" instance_seqno: 1794970498532688) as {username='slave'} at 127.2.229.65:57643; Asking this server to re-register.
I20261118 02:55:13.290880  4846 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" instance_seqno: 1794970499527172) as {username='slave'} at 127.2.229.67:34069; Asking this server to re-register.
I20261118 02:55:13.291088  3665 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:13.291548  3949 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:13.292868  4846 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:13.293329  4845 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:13.661460  4910 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:13.661701  4910 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:13.663705  4910 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:13.664698  4832 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:13.666074  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:13.666324  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:13.666428  4832 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:13.666774  4910 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 02:55:14.398300  4919 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:14.398475  4919 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:14.399085  4919 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:14.401381  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:14.401618  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:14.401734  4832 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:14.402002  4919 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 02:55:15.200219  4927 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:15.200452  4927 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:15.201141  4927 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:15.203652  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:15.203918  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:15.204056  4832 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:15.204376  4927 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20261118 02:55:16.395913  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.124:36695
--webserver_interface=127.2.229.124
--webserver_port=39557
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 02:55:16.440620  4948 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:16.440841  4948 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:16.441515  4948 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:16.443845  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:16.444074  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:16.444254  4832 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:16.444526  4948 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20261118 02:55:16.739959  4947 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:55:16.740352  4947 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:55:16.740473  4947 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:55:16.749022  4947 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:55:16.749122  4947 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:55:16.749184  4947 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:55:16.749251  4947 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:55:16.759747  4947 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.124:36695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.229.124
--webserver_port=39557
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:55:16.761472  4947 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:55:16.763316  4947 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:55:16.772500  4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:16.772330  4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:16.772840  4958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:16.774017  4947 server_base.cc:1061] running on GCE node
I20261118 02:55:16.774778  4947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:55:16.776278  4947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:55:16.777500  4947 hybrid_clock.cc:648] HybridClock initialized: now 1794970516777444 us; error 47 us; skew 500 ppm
I20261118 02:55:16.777848  4947 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:55:16.779922  4947 webserver.cc:577] Webserver started at http://127.2.229.124:39557/ using document root <none> and password file <none>
I20261118 02:55:16.780642  4947 fs_manager.cc:362] Metadata directory not provided
I20261118 02:55:16.780753  4947 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:55:16.786494  4947 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.003s	sys 0.002s
I20261118 02:55:16.789310  4965 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:55:16.791010  4947 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20261118 02:55:16.791225  4947 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
uuid: "813a6937cea442ef81645fccc32207f4"
format_stamp: "Formatted at 2026-11-18 02:54:57 on dist-test-slave-49p1"
I20261118 02:55:16.792274  4947 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:55:16.812065  4947 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:55:16.812987  4947 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:55:16.813256  4947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:55:16.831676  4947 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.124:36695
I20261118 02:55:16.831676  5018 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.124:36695 every 8 connection(s)
I20261118 02:55:16.833537  4947 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
I20261118 02:55:16.840436  5019 sys_catalog.cc:263] Verifying existing consensus state
I20261118 02:55:16.843278  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 4947
I20261118 02:55:16.843986  5019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap starting.
I20261118 02:55:16.862686  5019 log.cc:826] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:16.933223  5019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=97 overwritten=0 applied=97 ignored=0} inserts{seen=65 ignored=0} mutations{seen=83 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:16.933740  5019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap complete.
I20261118 02:55:16.938995  5019 raft_consensus.cc:360] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:16.939662  5019 raft_consensus.cc:748] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813a6937cea442ef81645fccc32207f4, State: Initialized, Role: FOLLOWER
I20261118 02:55:16.940313  5019 consensus_queue.cc:261] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 97, Last appended: 5.97, Last appended by leader: 97, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:16.941090  5023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:16.941354  5023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:16.942106  5019 sys_catalog.cc:565] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:55:16.955051  5037 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:16.955246  5037 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: loading cluster ID for follower catalog manager: success
I20261118 02:55:16.951622  4947 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:16.957418  4947 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:16.959854  5037 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: acquiring CA information for follower catalog manager: success
I20261118 02:55:16.963867  5037 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:55:17.268033  5042 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:17.268275  5042 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:17.268959  5042 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:17.272802  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:17.277618  4994 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 7 candidate_status { last_received { term: 6 index: 101 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4" is_pre_election: true
I20261118 02:55:17.278187  4994 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 457eb38cba564ecd9f6f11245172260b in term 5.
I20261118 02:55:17.278980  4832 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d
I20261118 02:55:17.279316  5042 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Leader pre-election won for term 7
I20261118 02:55:17.279435  5042 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:17.279528  5042 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 6 FOLLOWER]: Advancing to term 7
I20261118 02:55:17.283109  5042 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:17.283646  5042 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 election: Requested vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:17.284798  4994 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 7 candidate_status { last_received { term: 6 index: 101 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4"
I20261118 02:55:17.284991  4994 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 5 FOLLOWER]: Advancing to term 7
W20261118 02:55:17.285351  4832 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:17.289671  4994 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 457eb38cba564ecd9f6f11245172260b in term 7.
I20261118 02:55:17.290100  4832 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d
I20261118 02:55:17.290369  5042 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 7 FOLLOWER]: Leader election won for term 7
I20261118 02:55:17.290776  5042 raft_consensus.cc:705] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 7 LEADER]: Becoming Leader. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Running, Role: LEADER
I20261118 02:55:17.291271  5042 consensus_queue.cc:238] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 101, Committed index: 101, Last appended: 6.101, Last appended by leader: 101, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:17.293905  3664 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:55:17.298641  4984 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" instance_seqno: 1794970498532688) as {username='slave'} at 127.2.229.65:48463; Asking this server to re-register.
I20261118 02:55:17.302032  4994 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Refusing update from remote peer 457eb38cba564ecd9f6f11245172260b: Log matching property violated. Preceding OpId in replica: term: 5 index: 97. Preceding OpId from leader: term: 7 index: 102. (index mismatch)
W20261118 02:55:17.304693  4832 consensus_peers.cc:597] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b -> Peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Couldn't send request to peer 26c6b20fa203404a88463ea27bdd3c7d. Status: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:17.304706  5042 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Connected to new peer: Peer: permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 102, Last known committed idx: 97, Time since last communication: 0.000s
I20261118 02:55:17.318737  5047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 7 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:17.320247  5047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:17.325256  3664 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:17.328212  5056 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:55:17.335031  3948 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:55:17.335955  5056 catalog_manager.cc:694] Loaded metadata for table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]
I20261118 02:55:17.336545  5056 catalog_manager.cc:694] Loaded metadata for table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]
I20261118 02:55:17.342368  5056 catalog_manager.cc:694] Loaded metadata for table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]
I20261118 02:55:17.343005  5056 catalog_manager.cc:694] Loaded metadata for table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]
I20261118 02:55:17.344633  5056 catalog_manager.cc:694] Loaded metadata for table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]
I20261118 02:55:17.345050  5056 catalog_manager.cc:694] Loaded metadata for table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]
I20261118 02:55:17.345623  5056 catalog_manager.cc:694] Loaded metadata for table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da]
I20261118 02:55:17.346002  3801 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:55:17.346076  5056 catalog_manager.cc:694] Loaded metadata for table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
I20261118 02:55:17.346498  5056 catalog_manager.cc:694] Loaded metadata for table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]
I20261118 02:55:17.346930  5056 catalog_manager.cc:694] Loaded metadata for table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:17.347308  5056 catalog_manager.cc:694] Loaded metadata for table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:17.347702  5056 catalog_manager.cc:694] Loaded metadata for table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b]
I20261118 02:55:17.348191  5056 catalog_manager.cc:694] Loaded metadata for table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]
I20261118 02:55:17.348608  5056 catalog_manager.cc:694] Loaded metadata for table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]
I20261118 02:55:17.349020  5056 catalog_manager.cc:694] Loaded metadata for table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]
I20261118 02:55:17.349402  5056 catalog_manager.cc:694] Loaded metadata for table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]
I20261118 02:55:17.349769  5056 catalog_manager.cc:694] Loaded metadata for table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:17.350140  5056 catalog_manager.cc:694] Loaded metadata for table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]
I20261118 02:55:17.350515  5056 catalog_manager.cc:694] Loaded metadata for table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]
I20261118 02:55:17.350921  5056 catalog_manager.cc:694] Loaded metadata for table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]
I20261118 02:55:17.353554  5056 tablet_loader.cc:96] loaded metadata for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7])
I20261118 02:55:17.353971  5056 tablet_loader.cc:96] loaded metadata for tablet 0e901a03b90e4c978eb98be87163cfe5 (table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190])
I20261118 02:55:17.356763  5054 mvcc.cc:204] Tried to move back new op lower bound from 7352199215829995520 to 7352199215350587392. Current Snapshot: MvccSnapshot[applied={T|T < 7352199215829995520 or (T in {7352199215829995520})}]
I20261118 02:55:17.357405  5056 tablet_loader.cc:96] loaded metadata for tablet 1904140da78b4567a9decb250c6f74a3 (table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f])
I20261118 02:55:17.357864  5056 tablet_loader.cc:96] loaded metadata for tablet 1d347c0ae53241708ec623562f4740d4 (table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b])
I20261118 02:55:17.358242  5056 tablet_loader.cc:96] loaded metadata for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db])
I20261118 02:55:17.358572  5056 tablet_loader.cc:96] loaded metadata for tablet 2c92955aa65142f1bd20f4b8f3321500 (table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d])
I20261118 02:55:17.359119  5056 tablet_loader.cc:96] loaded metadata for tablet 46f05790dbdc4b8185a7b0362f604cf8 (table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf])
I20261118 02:55:17.360445  5056 tablet_loader.cc:96] loaded metadata for tablet 65bd6fcaed5b47c3a2478d375ace9636 (table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73])
I20261118 02:55:17.360956  5056 tablet_loader.cc:96] loaded metadata for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b])
I20261118 02:55:17.361289  5056 tablet_loader.cc:96] loaded metadata for tablet 78189cccd457466bb5d01231038fec24 (table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da])
I20261118 02:55:17.361581  5056 tablet_loader.cc:96] loaded metadata for tablet 8bed657d0da64d95851f8a86216b6c2c (table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c])
I20261118 02:55:17.362082  5056 tablet_loader.cc:96] loaded metadata for tablet 973831392ec24e0681c3d4fffd8bb199 (table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139])
I20261118 02:55:17.362761  5056 tablet_loader.cc:96] loaded metadata for tablet a013d180f2cd430ab1ba30b39fd27591 (table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d])
I20261118 02:55:17.363284  5056 tablet_loader.cc:96] loaded metadata for tablet ab61b43e639a4e75bc2e7009ea86a64f (table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67])
I20261118 02:55:17.363675  5056 tablet_loader.cc:96] loaded metadata for tablet b129518ce23c45758ebcaa55410757d8 (table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1])
I20261118 02:55:17.364313  5056 tablet_loader.cc:96] loaded metadata for tablet c0713dcf79f54873a99d7556ad8f1454 (table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec])
I20261118 02:55:17.364655  5056 tablet_loader.cc:96] loaded metadata for tablet c92c29c30ab6411dabe6f02331d29b7a (table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d])
I20261118 02:55:17.365108  5056 tablet_loader.cc:96] loaded metadata for tablet ccf78258c9634540a178855a142f4d4b (table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e])
I20261118 02:55:17.365415  5023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 7 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:17.365669  5023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:17.367105  4983 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:17.367910  5023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:17.368116  5023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:17.368207  5058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:17.368510  5058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:17.369226  4981 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" instance_seqno: 1794970499028671) as {username='slave'} at 127.2.229.66:34351; Asking this server to re-register.
I20261118 02:55:17.370055  4982 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" instance_seqno: 1794970499527172) as {username='slave'} at 127.2.229.67:36637; Asking this server to re-register.
I20261118 02:55:17.370177  3801 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:17.370800  3948 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:17.371964  5056 tablet_loader.cc:96] loaded metadata for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6])
I20261118 02:55:17.372258  4982 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:17.372856  5056 tablet_loader.cc:96] loaded metadata for tablet ffe8d4c111d14054b76f9c7724072a94 (table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66])
I20261118 02:55:17.373199  5056 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:55:17.373705  4982 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:17.374462  5053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:17.374694  5053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:17.374845  5056 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:17.374914  5056 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:55:17.376365  5056 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:55:17.377336  5056 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Loaded TSK: 0
I20261118 02:55:17.378203  5056 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:55:17.379172  5056 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:55:17.380203  5056 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 20
I20261118 02:55:17.478365  4845 master_service.cc:965] ReplaceTablet: received request to replace tablet c92c29c30ab6411dabe6f02331d29b7a from {username='slave'} at 127.0.0.1:42302
I20261118 02:55:17.478494  4846 master_service.cc:965] ReplaceTablet: received request to replace tablet 1d347c0ae53241708ec623562f4740d4 from {username='slave'} at 127.0.0.1:42302
I20261118 02:55:17.479507  4842 master_service.cc:965] ReplaceTablet: received request to replace tablet 1d347c0ae53241708ec623562f4740d4 from {username='slave'} at 127.0.0.1:42302
I20261118 02:55:17.479658  4840 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_87d592a09c3348028719a603956fbce6" } modify_external_catalogs: true
I20261118 02:55:17.480013  4840 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_87d592a09c3348028719a603956fbce6" } modify_external_catalogs: true
I20261118 02:55:17.479971  4839 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42302:
name: "default.Table_bdbed231860946c2bf9b514f724bc3e9"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:17.480463  4838 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42302:
name: "default.Table_40868189cecf45d0b705461c4328ee17"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:17.481050  4837 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42302:
name: "default.Table_52fca7a2e65f4e679bbef9de3df569f1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:17.482582  4841 master_service.cc:965] ReplaceTablet: received request to replace tablet 55f81e7b62e54ec18c7c13a200edb68f from {username='slave'} at 127.0.0.1:42302
I20261118 02:55:17.482751  4843 master_service.cc:965] ReplaceTablet: received request to replace tablet a013d180f2cd430ab1ba30b39fd27591 from {username='slave'} at 127.0.0.1:42302
I20261118 02:55:17.482508  4844 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_2e3917ff98f2474984580e61caa9c17b" } new_table_name: "default.Table_3c225da1f7fa435083909e053ec9751a" modify_external_catalogs: true
I20261118 02:55:17.483662  4841 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_571d1360811c42dc815efb439b2b52e6" } modify_external_catalogs: true
I20261118 02:55:17.483940  4841 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_571d1360811c42dc815efb439b2b52e6" } modify_external_catalogs: true
F20261118 02:55:17.488473  4898 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 15 received events: NotificationEvent(eventId=21, eventTime=1794970503, eventType=CREATE_TABLE, dbName=default, tableName=table_f0ae490a0df64e1097e144c7a370df61, message={"server":"","servicePrincipal":"","db":"default","table":"table_f0ae490a0df64e1097e144c7a370df61","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f0ae490a0df64e1097e144c7a370df61\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970503},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_f0ae490a0df64e1097e144c7a370df61\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f0ae490a0df64e1097e144c7a370df61\",\"kudu.table_id\":\"a59b9425c01049269883cf7767936c67\",\"transient_lastDdlTime\":\"1794970503\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1794970503, eventType=DROP_TABLE, dbName=default, tableName=table_c113c2b57e17479e92e241234a241af7, message={"server":"","servicePrincipal":"","db":"default","table":"table_c113c2b57e17479e92e241234a241af7","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c113c2b57e17479e92e241234a241af7\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970500},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_c113c2b57e17479e92e241234a241af7\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c113c2b57e17479e92e241234a241af7\",\"kudu.table_id\":\"da0cb25ccfed426ca61f0dd9f93d30ec\",\"transient_lastDdlTime\":\"1794970500\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
    @     0x7f55fe6eed5d  google::LogMessage::Fail() at ??:0
    @     0x7f55fe6f2b23  google::LogMessage::SendToLog() at ??:0
    @     0x7f55fe6ee75c  google::LogMessage::Flush() at ??:0
I20261118 02:55:17.497677  4845 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:17.497782  4842 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_28c62f411da74545aa42dad7325bc064" } modify_external_catalogs: true
I20261118 02:55:17.498061  4842 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_28c62f411da74545aa42dad7325bc064" } modify_external_catalogs: true
    @     0x7f55fe6ee9a1  google::LogMessage::~LogMessage() at ??:0
I20261118 02:55:17.499245  4846 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet 1d347c0ae53241708ec623562f4740d4
I20261118 02:55:17.499305  4843 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:17.501046  4843 catalog_manager.cc:6936] ReplaceTablet: tablet a013d180f2cd430ab1ba30b39fd27591 deleted and replaced by tablet 8ae679d2feb04b95ab2f4fb51e443d64
I20261118 02:55:17.501396  4846 catalog_manager.cc:6936] ReplaceTablet: tablet 1d347c0ae53241708ec623562f4740d4 deleted and replaced by tablet a67bc60c7ecb4b7e8e03e04267bb91a5
I20261118 02:55:17.501461  4843 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42302:
name: "default.Table_e39f59e2fdbb47fda4c10beba29ad8f1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:17.501746  4846 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_4f7796d5d5d54def843b971e66d15ea6" } new_table_name: "default.Table_79eb11797e3f4bc2bffe161096300f7a" modify_external_catalogs: true
    @     0x7f5613b6e9a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 02:55:17.508414  4845 catalog_manager.cc:6936] ReplaceTablet: tablet c92c29c30ab6411dabe6f02331d29b7a deleted and replaced by tablet b82ca8a282f94392bb2746ed7b8b8ba7
I20261118 02:55:17.509038  4845 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" } modify_external_catalogs: true
I20261118 02:55:17.509253  4845 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42302:
table { table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" } modify_external_catalogs: true
    @     0x7f5613b6d251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 02:55:17.524923  3734 tablet_service.cc:1547] Processing DeleteTablet for tablet c92c29c30ab6411dabe6f02331d29b7a with delete_type TABLET_DATA_DELETED (replaced by tablet b82ca8a282f94392bb2746ed7b8b8ba7) from {username='slave'} at 127.0.0.1:41072
    @     0x7f5613b74ecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 02:55:17.528736  3733 tablet_service.cc:1547] Processing DeleteTablet for tablet a013d180f2cd430ab1ba30b39fd27591 with delete_type TABLET_DATA_DELETED (replaced by tablet 8ae679d2feb04b95ab2f4fb51e443d64) from {username='slave'} at 127.0.0.1:41072
I20261118 02:55:17.529330  3733 tablet_service.cc:1547] Processing DeleteTablet for tablet 1d347c0ae53241708ec623562f4740d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a67bc60c7ecb4b7e8e03e04267bb91a5) from {username='slave'} at 127.0.0.1:41072
I20261118 02:55:17.529987  3733 tablet_service.cc:1500] Processing CreateTablet for tablet 8ae679d2feb04b95ab2f4fb51e443d64 (DEFAULT_TABLE table=default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:17.530427  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae679d2feb04b95ab2f4fb51e443d64. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f5613b74d49  std::_Function_handler<>::_M_invoke() at ??:0
I20261118 02:55:17.530959  5075 tablet_replica.cc:333] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:17.531199  5075 raft_consensus.cc:2261] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 02:55:17.531463  5075 raft_consensus.cc:2290] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:17.532120  5075 ts_tablet_manager.cc:1962] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
    @     0x7f55ffb37dc9  kudu::Thread::SuperviseThread() at ??:0
    @     0x7f56032126db  start_thread at ??:0
I20261118 02:55:17.537914  3734 tablet_service.cc:1500] Processing CreateTablet for tablet a67bc60c7ecb4b7e8e03e04267bb91a5 (DEFAULT_TABLE table=default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:17.538292  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67bc60c7ecb4b7e8e03e04267bb91a5. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f55fbc8261f  clone at ??:0
I20261118 02:55:17.539463  5075 ts_tablet_manager.cc:1975] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:17.539587  5075 log.cc:1199] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:17.539908  5075 ts_tablet_manager.cc:1996] T c92c29c30ab6411dabe6f02331d29b7a P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:17.540879  5075 tablet_replica.cc:333] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:17.541091  5075 raft_consensus.cc:2261] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:17.541401  5075 raft_consensus.cc:2290] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:17.541858  5075 ts_tablet_manager.cc:1962] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:17.542951  3732 tablet_service.cc:1500] Processing CreateTablet for tablet b82ca8a282f94392bb2746ed7b8b8ba7 (DEFAULT_TABLE table=default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:17.543397  3732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82ca8a282f94392bb2746ed7b8b8ba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:17.545748  5077 tablet_bootstrap.cc:492] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:17.545882  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet a013d180f2cd430ab1ba30b39fd27591 with delete_type TABLET_DATA_DELETED (replaced by tablet 8ae679d2feb04b95ab2f4fb51e443d64) from {username='slave'} at 127.0.0.1:36196
I20261118 02:55:17.546530  3877 tablet_service.cc:1500] Processing CreateTablet for tablet b82ca8a282f94392bb2746ed7b8b8ba7 (DEFAULT_TABLE table=default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:17.547045  3877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82ca8a282f94392bb2746ed7b8b8ba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:17.547571  5077 tablet_bootstrap.cc:654] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:17.548264  3879 tablet_service.cc:1547] Processing DeleteTablet for tablet c92c29c30ab6411dabe6f02331d29b7a with delete_type TABLET_DATA_DELETED (replaced by tablet b82ca8a282f94392bb2746ed7b8b8ba7) from {username='slave'} at 127.0.0.1:36196
I20261118 02:55:17.548484  3882 tablet_service.cc:1547] Processing DeleteTablet for tablet 1d347c0ae53241708ec623562f4740d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a67bc60c7ecb4b7e8e03e04267bb91a5) from {username='slave'} at 127.0.0.1:36196
I20261118 02:55:17.548853  3880 tablet_service.cc:1500] Processing CreateTablet for tablet 8ae679d2feb04b95ab2f4fb51e443d64 (DEFAULT_TABLE table=default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:17.549240  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae679d2feb04b95ab2f4fb51e443d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:17.555918  3878 tablet_service.cc:1500] Processing CreateTablet for tablet a67bc60c7ecb4b7e8e03e04267bb91a5 (DEFAULT_TABLE table=default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:17.556412  3878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67bc60c7ecb4b7e8e03e04267bb91a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:17.573114  5075 ts_tablet_manager.cc:1975] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:17.573272  5075 log.cc:1199] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:17.573688  5075 ts_tablet_manager.cc:1996] T a013d180f2cd430ab1ba30b39fd27591 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:17.574684  5075 tablet_replica.cc:333] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:17.574885  5075 raft_consensus.cc:2261] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:17.575067  5075 raft_consensus.cc:2290] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:17.575676  5075 ts_tablet_manager.cc:1962] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:17.577378  5080 tablet_bootstrap.cc:492] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:17.579586  5080 tablet_bootstrap.cc:654] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:17.587100  5075 ts_tablet_manager.cc:1975] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:17.587204  5075 log.cc:1199] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/1d347c0ae53241708ec623562f4740d4
I20261118 02:55:17.587567  5075 ts_tablet_manager.cc:1996] T 1d347c0ae53241708ec623562f4740d4 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:17.589464  5080 tablet_bootstrap.cc:492] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:17.589607  5080 ts_tablet_manager.cc:1449] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.012s	user 0.001s	sys 0.002s
I20261118 02:55:17.590394  5080 raft_consensus.cc:360] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.590595  5080 raft_consensus.cc:386] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:17.590677  5080 raft_consensus.cc:748] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:17.591002  5080 consensus_queue.cc:261] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.592978  5078 tablet_replica.cc:333] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:17.593180  5078 raft_consensus.cc:2261] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:17.593397  5078 raft_consensus.cc:2290] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:17.593942  5078 ts_tablet_manager.cc:1962] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:17.597049  5080 ts_tablet_manager.cc:1480] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.002s
I20261118 02:55:17.600358  5078 ts_tablet_manager.cc:1975] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:17.600457  5078 log.cc:1199] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:17.600832  5078 ts_tablet_manager.cc:1996] T a013d180f2cd430ab1ba30b39fd27591 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:17.601270  5080 tablet_bootstrap.cc:492] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:17.601788  5078 tablet_replica.cc:333] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:17.602504  5078 raft_consensus.cc:2261] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:17.602813  5078 raft_consensus.cc:2290] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:17.603102  5080 tablet_bootstrap.cc:654] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:17.603363  5078 ts_tablet_manager.cc:1962] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:17.607169  5080 tablet_bootstrap.cc:492] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:17.607319  5080 ts_tablet_manager.cc:1449] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.006s	user 0.000s	sys 0.005s
I20261118 02:55:17.607362  5077 tablet_bootstrap.cc:492] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:17.607503  5077 ts_tablet_manager.cc:1449] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.062s	user 0.007s	sys 0.046s
I20261118 02:55:17.608132  5080 raft_consensus.cc:360] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.608337  5080 raft_consensus.cc:386] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:17.608412  5080 raft_consensus.cc:748] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:17.608515  5077 raft_consensus.cc:360] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.608704  5077 raft_consensus.cc:386] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:17.608784  5077 raft_consensus.cc:748] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:17.608922  5080 consensus_queue.cc:261] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.609016  5077 consensus_queue.cc:261] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.609579  5077 ts_tablet_manager.cc:1480] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.002s
I20261118 02:55:17.609869  5077 tablet_bootstrap.cc:492] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:17.610651  5078 ts_tablet_manager.cc:1975] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:17.610762  5078 log.cc:1199] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:17.611131  5078 ts_tablet_manager.cc:1996] T c92c29c30ab6411dabe6f02331d29b7a P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:17.611649  5077 tablet_bootstrap.cc:654] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:17.611940  5080 ts_tablet_manager.cc:1480] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.002s
I20261118 02:55:17.612023  5078 tablet_replica.cc:333] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:17.612258  5078 raft_consensus.cc:2261] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:17.612262  5080 tablet_bootstrap.cc:492] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:17.612458  5078 raft_consensus.cc:2290] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:17.612984  5078 ts_tablet_manager.cc:1962] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:17.614028  5080 tablet_bootstrap.cc:654] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:17.619131  5078 ts_tablet_manager.cc:1975] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:17.619253  5078 log.cc:1199] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/1d347c0ae53241708ec623562f4740d4
I20261118 02:55:17.619630  5078 ts_tablet_manager.cc:1996] T 1d347c0ae53241708ec623562f4740d4 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:17.620595  5077 tablet_bootstrap.cc:492] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:17.621766  5077 ts_tablet_manager.cc:1449] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.012s	user 0.003s	sys 0.000s
I20261118 02:55:17.622485  5077 raft_consensus.cc:360] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.622645  5080 tablet_bootstrap.cc:492] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:17.622658  5077 raft_consensus.cc:386] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:17.622720  5077 raft_consensus.cc:748] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:17.622758  5080 ts_tablet_manager.cc:1449] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.011s	user 0.002s	sys 0.001s
I20261118 02:55:17.622977  5077 consensus_queue.cc:261] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.623397  5080 raft_consensus.cc:360] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.623488  5077 ts_tablet_manager.cc:1480] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20261118 02:55:17.623556  5080 raft_consensus.cc:386] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:17.623627  5080 raft_consensus.cc:748] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:17.623852  5080 consensus_queue.cc:261] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.626335  5080 ts_tablet_manager.cc:1480] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.003s	user 0.000s	sys 0.001s
I20261118 02:55:17.627665  4612 raft_consensus.cc:499] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:17.627800  4612 raft_consensus.cc:521] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.628818  4612 leader_election.cc:290] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:17.629498  5077 tablet_bootstrap.cc:492] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:17.629948  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a67bc60c7ecb4b7e8e03e04267bb91a5" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
W20261118 02:55:17.629972  3691 leader_election.cc:343] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: a67bc60c7ecb4b7e8e03e04267bb91a5
I20261118 02:55:17.630139  3901 raft_consensus.cc:2490] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:17.630614  3690 leader_election.cc:304] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:17.630906  4612 raft_consensus.cc:2826] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:17.631014  4612 raft_consensus.cc:499] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:17.631106  4612 raft_consensus.cc:3096] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:17.631256  5077 tablet_bootstrap.cc:654] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:17.634172  4612 raft_consensus.cc:521] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.634842  5077 tablet_bootstrap.cc:492] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:17.634900  4612 leader_election.cc:290] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:17.635076  5077 ts_tablet_manager.cc:1449] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.011s	user 0.002s	sys 0.003s
I20261118 02:55:17.635357  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "a67bc60c7ecb4b7e8e03e04267bb91a5" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:17.635545  3901 raft_consensus.cc:3096] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:17.635860  5077 raft_consensus.cc:360] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.636040  5077 raft_consensus.cc:386] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20261118 02:55:17.635906  3691 leader_election.cc:343] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: a67bc60c7ecb4b7e8e03e04267bb91a5
I20261118 02:55:17.636250  5077 raft_consensus.cc:748] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:17.636503  5077 consensus_queue.cc:261] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.637009  5077 ts_tablet_manager.cc:1480] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.002s	user 0.000s	sys 0.001s
I20261118 02:55:17.638312  3901 raft_consensus.cc:2490] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:17.638796  3690 leader_election.cc:304] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:17.639231  4612 raft_consensus.cc:2826] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:17.639664  4612 raft_consensus.cc:705] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:17.639998  4612 consensus_queue.cc:238] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.642830  5090 consensus_queue.cc:1049] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 02:55:17.643293  3691 consensus_peers.cc:597] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a67bc60c7ecb4b7e8e03e04267bb91a5. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:17.740816  4612 raft_consensus.cc:499] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:17.741046  4612 raft_consensus.cc:521] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.741828  4612 leader_election.cc:290] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:17.742376  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b82ca8a282f94392bb2746ed7b8b8ba7" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:17.742729  3901 raft_consensus.cc:2490] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
W20261118 02:55:17.742722  3691 leader_election.cc:343] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7
I20261118 02:55:17.743161  3690 leader_election.cc:304] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:17.743471  4612 raft_consensus.cc:2826] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:17.743564  4612 raft_consensus.cc:499] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:17.743664  4612 raft_consensus.cc:3096] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:17.745707  4612 raft_consensus.cc:521] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.746248  4612 leader_election.cc:290] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:17.746776  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b82ca8a282f94392bb2746ed7b8b8ba7" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:17.746963  3901 raft_consensus.cc:3096] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
W20261118 02:55:17.747074  3691 leader_election.cc:343] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7
I20261118 02:55:17.749910  3901 raft_consensus.cc:2490] T b82ca8a282f94392bb2746ed7b8b8ba7 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:17.750299  3690 leader_election.cc:304] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:17.750545  4612 raft_consensus.cc:2826] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:17.750725  4612 raft_consensus.cc:705] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:17.750998  4612 consensus_queue.cc:238] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:17.754232  4612 consensus_queue.cc:1049] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 02:55:17.754529  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 1: this message will repeat every 5th retry.
W20261118 02:55:17.777125  3690 connection.cc:573] server connection from 127.0.0.1:41072 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:17.777125  3835 connection.cc:573] server connection from 127.0.0.1:36196 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:17.777477  3836 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:17.777351  5069 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:17.504693 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.2.229.65:33705)
1118 02:55:17.530605 (+ 25912us) server_negotiation.cc:207] Beginning negotiation
1118 02:55:17.530611 (+     6us) server_negotiation.cc:400] Waiting for connection header
1118 02:55:17.530630 (+    19us) server_negotiation.cc:408] Connection header received
1118 02:55:17.530705 (+    75us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
1118 02:55:17.530710 (+     5us) server_negotiation.cc:462] Received NEGOTIATE request from client
1118 02:55:17.530781 (+    71us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
1118 02:55:17.531519 (+   738us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
1118 02:55:17.531760 (+   241us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
1118 02:55:17.533236 (+  1476us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
1118 02:55:17.777302 (+244066us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46584: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"server-negotiator.queue_time_us":25783,"thread_start_us":25679,"threads_started":1}
W20261118 02:55:17.778443  3689 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:17.779369  3955 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:17.780386  3955 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.229.125:38339)
W20261118 02:55:17.780925  3955 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.229.125:38339)
W20261118 02:55:17.781210  3955 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.2.229.125:38339)
W20261118 02:55:17.781391  3955 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.2.229.125:38339)
W20261118 02:55:17.880867  3553 consensus_peers.cc:597] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 -> Peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Couldn't send request to peer 3f1973986ec1466eab5bec01dd81654f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d347c0ae53241708ec623562f4740d4. This is attempt 1: this message will repeat every 5th retry.
W20261118 02:55:17.976025  3552 consensus_peers.cc:597] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 -> Peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Couldn't send request to peer 99a39b1d9d814deeb3a8a644e9b392ef. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d347c0ae53241708ec623562f4740d4. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:18.019960  5100 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:18.020197  5100 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:18.021987  5100 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:18.022921  4970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:18.024148  4969 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:18.024335  4970 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:18.024495  4970 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:18.024865  5100 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 02:55:18.190639  4612 raft_consensus.cc:499] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:18.190848  4612 raft_consensus.cc:521] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:18.191565  4612 leader_election.cc:290] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:18.192157  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8ae679d2feb04b95ab2f4fb51e443d64" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:18.192366  3902 raft_consensus.cc:2490] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
W20261118 02:55:18.192399  3691 leader_election.cc:343] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64
I20261118 02:55:18.192745  3690 leader_election.cc:304] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:18.193009  4612 raft_consensus.cc:2826] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:18.193091  4612 raft_consensus.cc:499] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:18.193192  4612 raft_consensus.cc:3096] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:18.195111  4612 raft_consensus.cc:521] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:18.195698  4612 leader_election.cc:290] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:18.196218  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "8ae679d2feb04b95ab2f4fb51e443d64" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:18.196403  3902 raft_consensus.cc:3096] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
W20261118 02:55:18.196491  3691 leader_election.cc:343] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64
I20261118 02:55:18.198383  3902 raft_consensus.cc:2490] T 8ae679d2feb04b95ab2f4fb51e443d64 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:18.198832  3690 leader_election.cc:304] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:18.199155  4612 raft_consensus.cc:2826] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:18.199343  4612 raft_consensus.cc:705] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:18.199625  4612 consensus_queue.cc:238] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:18.202332  4612 consensus_queue.cc:1049] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 02:55:18.202572  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:18.514424  5108 ts_tablet_manager.cc:979] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Initiating tablet copy from peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:18.515405  5108 tablet_copy_client.cc:323] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Beginning tablet copy session from remote peer at address 127.2.229.66:40377
I20261118 02:55:18.516284  3775 tablet_copy_service.cc:140] P 99a39b1d9d814deeb3a8a644e9b392ef: Received BeginTabletCopySession request for tablet b82ca8a282f94392bb2746ed7b8b8ba7 from peer 1fdf7d7ece324fefb3a0c376add5b7e8 ({username='slave'} at 127.2.229.65:55185)
I20261118 02:55:18.516482  3775 tablet_copy_service.cc:161] P 99a39b1d9d814deeb3a8a644e9b392ef: Beginning new tablet copy session on tablet b82ca8a282f94392bb2746ed7b8b8ba7 from peer 1fdf7d7ece324fefb3a0c376add5b7e8 at {username='slave'} at 127.2.229.65:55185: session id = 1fdf7d7ece324fefb3a0c376add5b7e8-b82ca8a282f94392bb2746ed7b8b8ba7
I20261118 02:55:18.518481  3775 tablet_copy_source_session.cc:215] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef: Tablet Copy: opened 0 blocks and 1 log segments
I20261118 02:55:18.520277  5108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b82ca8a282f94392bb2746ed7b8b8ba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:18.525393  5108 tablet_copy_client.cc:806] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Starting download of 0 data blocks...
I20261118 02:55:18.525678  5108 tablet_copy_client.cc:670] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Starting download of 1 WAL segments...
I20261118 02:55:18.527612  5108 tablet_copy_client.cc:538] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 02:55:18.530397  5108 tablet_bootstrap.cc:492] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:18.530742  5110 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:18.530900  5110 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:18.531637  5110 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:18.533679  4970 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:18.534319  4969 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:18.534439  4969 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:18.534683  5110 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 02:55:18.536306  5108 tablet_bootstrap.cc:492] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:18.536657  5108 tablet_bootstrap.cc:492] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap complete.
I20261118 02:55:18.536934  5108 ts_tablet_manager.cc:1449] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.007s	user 0.004s	sys 0.004s
I20261118 02:55:18.537604  5108 raft_consensus.cc:360] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:18.537762  5108 raft_consensus.cc:748] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:18.538019  5108 consensus_queue.cc:261] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:18.539575  5108 ts_tablet_manager.cc:1480] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.000s
I20261118 02:55:18.540712  3775 tablet_copy_service.cc:342] P 99a39b1d9d814deeb3a8a644e9b392ef: Request end of tablet copy session 1fdf7d7ece324fefb3a0c376add5b7e8-b82ca8a282f94392bb2746ed7b8b8ba7 received from {username='slave'} at 127.2.229.65:55185
I20261118 02:55:18.540854  3775 tablet_copy_service.cc:434] P 99a39b1d9d814deeb3a8a644e9b392ef: ending tablet copy session 1fdf7d7ece324fefb3a0c376add5b7e8-b82ca8a282f94392bb2746ed7b8b8ba7 on tablet b82ca8a282f94392bb2746ed7b8b8ba7 with peer 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:18.781210  5108 ts_tablet_manager.cc:979] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Initiating tablet copy from peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:18.781842  5108 tablet_copy_client.cc:323] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Beginning tablet copy session from remote peer at address 127.2.229.66:40377
I20261118 02:55:18.782650  3775 tablet_copy_service.cc:140] P 99a39b1d9d814deeb3a8a644e9b392ef: Received BeginTabletCopySession request for tablet a67bc60c7ecb4b7e8e03e04267bb91a5 from peer 1fdf7d7ece324fefb3a0c376add5b7e8 ({username='slave'} at 127.2.229.65:55185)
I20261118 02:55:18.782800  3775 tablet_copy_service.cc:161] P 99a39b1d9d814deeb3a8a644e9b392ef: Beginning new tablet copy session on tablet a67bc60c7ecb4b7e8e03e04267bb91a5 from peer 1fdf7d7ece324fefb3a0c376add5b7e8 at {username='slave'} at 127.2.229.65:55185: session id = 1fdf7d7ece324fefb3a0c376add5b7e8-a67bc60c7ecb4b7e8e03e04267bb91a5
I20261118 02:55:18.784861  3775 tablet_copy_source_session.cc:215] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 99a39b1d9d814deeb3a8a644e9b392ef: Tablet Copy: opened 0 blocks and 1 log segments
I20261118 02:55:18.786069  5108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a67bc60c7ecb4b7e8e03e04267bb91a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:18.790584  5108 tablet_copy_client.cc:806] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Starting download of 0 data blocks...
I20261118 02:55:18.790864  5108 tablet_copy_client.cc:670] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Starting download of 1 WAL segments...
I20261118 02:55:18.792666  5108 tablet_copy_client.cc:538] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 02:55:18.795193  5108 tablet_bootstrap.cc:492] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:18.799563  5108 tablet_bootstrap.cc:492] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:18.799887  5108 tablet_bootstrap.cc:492] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap complete.
I20261118 02:55:18.800124  5108 ts_tablet_manager.cc:1449] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.005s	user 0.005s	sys 0.000s
I20261118 02:55:18.800774  5108 raft_consensus.cc:360] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:18.800941  5108 raft_consensus.cc:748] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:18.801184  5108 consensus_queue.cc:261] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:18.801667  5108 ts_tablet_manager.cc:1480] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20261118 02:55:18.802863  3775 tablet_copy_service.cc:342] P 99a39b1d9d814deeb3a8a644e9b392ef: Request end of tablet copy session 1fdf7d7ece324fefb3a0c376add5b7e8-a67bc60c7ecb4b7e8e03e04267bb91a5 received from {username='slave'} at 127.2.229.65:55185
I20261118 02:55:18.803018  3775 tablet_copy_service.cc:434] P 99a39b1d9d814deeb3a8a644e9b392ef: ending tablet copy session 1fdf7d7ece324fefb3a0c376add5b7e8-a67bc60c7ecb4b7e8e03e04267bb91a5 on tablet a67bc60c7ecb4b7e8e03e04267bb91a5 with peer 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:19.165515  5108 ts_tablet_manager.cc:979] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Initiating tablet copy from peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:19.166119  5108 tablet_copy_client.cc:323] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Beginning tablet copy session from remote peer at address 127.2.229.66:40377
I20261118 02:55:19.166812  3775 tablet_copy_service.cc:140] P 99a39b1d9d814deeb3a8a644e9b392ef: Received BeginTabletCopySession request for tablet 8ae679d2feb04b95ab2f4fb51e443d64 from peer 1fdf7d7ece324fefb3a0c376add5b7e8 ({username='slave'} at 127.2.229.65:55185)
I20261118 02:55:19.166988  3775 tablet_copy_service.cc:161] P 99a39b1d9d814deeb3a8a644e9b392ef: Beginning new tablet copy session on tablet 8ae679d2feb04b95ab2f4fb51e443d64 from peer 1fdf7d7ece324fefb3a0c376add5b7e8 at {username='slave'} at 127.2.229.65:55185: session id = 1fdf7d7ece324fefb3a0c376add5b7e8-8ae679d2feb04b95ab2f4fb51e443d64
I20261118 02:55:19.169787  3775 tablet_copy_source_session.cc:215] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef: Tablet Copy: opened 0 blocks and 1 log segments
I20261118 02:55:19.171038  5108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ae679d2feb04b95ab2f4fb51e443d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:19.175735  5108 tablet_copy_client.cc:806] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Starting download of 0 data blocks...
I20261118 02:55:19.176014  5108 tablet_copy_client.cc:670] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Starting download of 1 WAL segments...
I20261118 02:55:19.177559  5108 tablet_copy_client.cc:538] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20261118 02:55:19.179965  5108 tablet_bootstrap.cc:492] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:19.183944  5108 tablet_bootstrap.cc:492] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:19.184286  5108 tablet_bootstrap.cc:492] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap complete.
I20261118 02:55:19.184455  5108 ts_tablet_manager.cc:1449] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.005s	user 0.005s	sys 0.001s
I20261118 02:55:19.185002  5108 raft_consensus.cc:360] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:19.185158  5108 raft_consensus.cc:748] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:19.185338  5108 consensus_queue.cc:261] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:19.185760  5108 ts_tablet_manager.cc:1480] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.000s
I20261118 02:55:19.187186  3775 tablet_copy_service.cc:342] P 99a39b1d9d814deeb3a8a644e9b392ef: Request end of tablet copy session 1fdf7d7ece324fefb3a0c376add5b7e8-8ae679d2feb04b95ab2f4fb51e443d64 received from {username='slave'} at 127.2.229.65:55185
I20261118 02:55:19.187378  3775 tablet_copy_service.cc:434] P 99a39b1d9d814deeb3a8a644e9b392ef: ending tablet copy session 1fdf7d7ece324fefb3a0c376add5b7e8-8ae679d2feb04b95ab2f4fb51e443d64 on tablet 8ae679d2feb04b95ab2f4fb51e443d64 with peer 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:19.260597  5132 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:19.260775  5132 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:19.261348  5132 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:19.263741  4969 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:19.263900  4970 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:19.264035  4970 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:19.264321  5132 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 02:55:19.728300  3616 raft_consensus.cc:1227] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1]   Dedup: 1.1->[]
I20261118 02:55:19.918701  3616 raft_consensus.cc:1227] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1]   Dedup: 1.1->[]
I20261118 02:55:19.946965  3616 raft_consensus.cc:1227] T a67bc60c7ecb4b7e8e03e04267bb91a5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 0.0->[1.1-1.1]   Dedup: 1.1->[]
I20261118 02:55:19.960668  5144 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:55:19.960837  5144 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:19.961663  5144 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:19.962544  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:55:19.962762  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:19.962883  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:19.963148  5144 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:20.042069  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.126:38025
--webserver_interface=127.2.229.126
--webserver_port=44955
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 02:55:20.120795  5143 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:55:20.120985  5143 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:20.121806  5143 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:20.122754  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:55:20.122915  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:20.123008  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:20.123252  5143 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:55:20.398454  5146 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:55:20.398861  5146 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:55:20.398989  5146 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:55:20.407805  5146 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:55:20.407904  5146 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:55:20.407989  5146 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:55:20.408149  5146 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:55:20.420967  5146 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.126:38025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.229.126
--webserver_port=44955
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:55:20.422909  5146 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:55:20.424867  5146 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:55:20.433331  5153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:20.433562  5154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:20.434264  5156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:20.435442  5146 server_base.cc:1061] running on GCE node
I20261118 02:55:20.436265  5146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:55:20.437827  5146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:55:20.439064  5146 hybrid_clock.cc:648] HybridClock initialized: now 1794970520438992 us; error 49 us; skew 500 ppm
I20261118 02:55:20.439438  5146 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:55:20.444727  5146 webserver.cc:577] Webserver started at http://127.2.229.126:44955/ using document root <none> and password file <none>
I20261118 02:55:20.445528  5146 fs_manager.cc:362] Metadata directory not provided
I20261118 02:55:20.445631  5146 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:55:20.452905  5146 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.004s	sys 0.001s
I20261118 02:55:20.456629  5164 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:55:20.458259  5146 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20261118 02:55:20.458413  5146 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
uuid: "26c6b20fa203404a88463ea27bdd3c7d"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:55:20.459086  5146 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:55:20.504082  5146 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:55:20.505003  5146 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:55:20.505275  5146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:55:20.525074  5217 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.126:38025 every 8 connection(s)
I20261118 02:55:20.525087  5146 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.126:38025
I20261118 02:55:20.526829  5146 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-0/data/info.pb
I20261118 02:55:20.533906  5218 sys_catalog.cc:263] Verifying existing consensus state
I20261118 02:55:20.534224  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 5146
I20261118 02:55:20.538407  5218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Bootstrap starting.
I20261118 02:55:20.564605  5218 log.cc:826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:20.637187  3663 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:55:20.637392  3663 heartbeater.cc:500] Master 127.2.229.126:38025 was elected leader, sending a full tablet report...
I20261118 02:55:20.644713  5218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=101 overwritten=0 applied=101 ignored=0} inserts{seen=67 ignored=0} mutations{seen=87 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:20.645326  5218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Bootstrap complete.
I20261118 02:55:20.650624  5218 raft_consensus.cc:360] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:20.651289  5218 raft_consensus.cc:748] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Initialized, Role: FOLLOWER
I20261118 02:55:20.651968  5218 consensus_queue.cc:261] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 101, Last appended: 6.101, Last appended by leader: 101, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:20.652774  5225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:20.653012  5225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:20.653786  5218 sys_catalog.cc:565] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:55:20.666072  5239 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:20.666260  5239 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: loading cluster ID for follower catalog manager: success
I20261118 02:55:20.671509  5239 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: acquiring CA information for follower catalog manager: success
I20261118 02:55:20.674789  5239 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:55:20.685365  5146 master_runner.cc:187] Error getting master registration for 127.2.229.125:38339: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111), 
I20261118 02:55:20.689010  5146 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:20.720813  3947 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:55:20.723443  5183 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" instance_seqno: 1794970499527172) as {username='slave'} at 127.2.229.67:39849; Asking this server to re-register.
I20261118 02:55:20.724186  3947 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:20.726233  5183 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:20.740029  5245 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:20.740310  5245 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:20.741003  5245 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:20.743934  4969 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:20.748364  5193 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 8 candidate_status { last_received { term: 7 index: 106 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:55:20.748924  5193 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 813a6937cea442ef81645fccc32207f4 in term 6.
I20261118 02:55:20.749716  4970 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4; no voters: 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:20.750082  5245 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20261118 02:55:20.750213  5245 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:20.750319  5245 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 7 FOLLOWER]: Advancing to term 8
I20261118 02:55:20.753501  5245 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:20.754292  5245 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 election: Requested vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
I20261118 02:55:20.754891  5193 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "813a6937cea442ef81645fccc32207f4" candidate_term: 8 candidate_status { last_received { term: 7 index: 106 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d"
I20261118 02:55:20.755136  5193 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 6 FOLLOWER]: Advancing to term 8
W20261118 02:55:20.755834  4969 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:20.759920  5193 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 813a6937cea442ef81645fccc32207f4 in term 8.
I20261118 02:55:20.760392  4970 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4; no voters: 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:20.760735  5245 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 8 FOLLOWER]: Leader election won for term 8
I20261118 02:55:20.761154  5245 raft_consensus.cc:705] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 8 LEADER]: Becoming Leader. State: Replica: 813a6937cea442ef81645fccc32207f4, State: Running, Role: LEADER
I20261118 02:55:20.761704  5245 consensus_queue.cc:238] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 106, Committed index: 106, Last appended: 7.106, Last appended by leader: 106, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:20.763628  5249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 813a6937cea442ef81645fccc32207f4. Latest consensus state: current_term: 8 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:20.763988  5249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: LEADER
W20261118 02:55:20.766536  4969 consensus_peers.cc:597] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 -> Peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Couldn't send request to peer 457eb38cba564ecd9f6f11245172260b. Status: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:20.767616  5193 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Refusing update from remote peer 813a6937cea442ef81645fccc32207f4: Log matching property violated. Preceding OpId in replica: term: 6 index: 101. Preceding OpId from leader: term: 8 index: 107. (index mismatch)
I20261118 02:55:20.768399  5245 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 107, Last known committed idx: 101, Time since last communication: 0.000s
I20261118 02:55:20.769163  5252 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:55:20.771977  5252 catalog_manager.cc:694] Loaded metadata for table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]
I20261118 02:55:20.772516  5252 catalog_manager.cc:694] Loaded metadata for table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]
I20261118 02:55:20.773025  5252 catalog_manager.cc:694] Loaded metadata for table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]
I20261118 02:55:20.773561  5252 catalog_manager.cc:694] Loaded metadata for table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]
I20261118 02:55:20.774039  5252 catalog_manager.cc:694] Loaded metadata for table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]
I20261118 02:55:20.774552  5252 catalog_manager.cc:694] Loaded metadata for table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]
I20261118 02:55:20.775027  5252 catalog_manager.cc:694] Loaded metadata for table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da]
I20261118 02:55:20.775450  5252 catalog_manager.cc:694] Loaded metadata for table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
I20261118 02:55:20.775779  5252 catalog_manager.cc:694] Loaded metadata for table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]
I20261118 02:55:20.776177  5252 catalog_manager.cc:694] Loaded metadata for table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:20.776612  5252 catalog_manager.cc:694] Loaded metadata for table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:20.777097  5252 catalog_manager.cc:694] Loaded metadata for table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b]
I20261118 02:55:20.777577  5252 catalog_manager.cc:694] Loaded metadata for table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]
I20261118 02:55:20.778100  5252 catalog_manager.cc:694] Loaded metadata for table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]
I20261118 02:55:20.778455  5252 catalog_manager.cc:694] Loaded metadata for table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]
I20261118 02:55:20.778784  5252 catalog_manager.cc:694] Loaded metadata for table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]
I20261118 02:55:20.779155  5252 catalog_manager.cc:694] Loaded metadata for table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:20.779510  5252 catalog_manager.cc:694] Loaded metadata for table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]
I20261118 02:55:20.779874  5252 catalog_manager.cc:694] Loaded metadata for table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]
I20261118 02:55:20.780372  5252 catalog_manager.cc:694] Loaded metadata for table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]
I20261118 02:55:20.783121  5252 tablet_loader.cc:96] loaded metadata for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7])
I20261118 02:55:20.783512  5252 tablet_loader.cc:96] loaded metadata for tablet 0e901a03b90e4c978eb98be87163cfe5 (table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190])
I20261118 02:55:20.783813  5252 tablet_loader.cc:96] loaded metadata for tablet 1904140da78b4567a9decb250c6f74a3 (table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f])
I20261118 02:55:20.784343  5252 tablet_loader.cc:96] loaded metadata for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db])
I20261118 02:55:20.784638  5252 tablet_loader.cc:96] loaded metadata for tablet 2c92955aa65142f1bd20f4b8f3321500 (table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d])
I20261118 02:55:20.785084  5252 tablet_loader.cc:96] loaded metadata for tablet 46f05790dbdc4b8185a7b0362f604cf8 (table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf])
I20261118 02:55:20.786100  5252 tablet_loader.cc:96] loaded metadata for tablet 65bd6fcaed5b47c3a2478d375ace9636 (table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73])
I20261118 02:55:20.786552  5252 tablet_loader.cc:96] loaded metadata for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b])
I20261118 02:55:20.786835  5252 tablet_loader.cc:96] loaded metadata for tablet 78189cccd457466bb5d01231038fec24 (table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da])
I20261118 02:55:20.787108  5252 tablet_loader.cc:96] loaded metadata for tablet 8ae679d2feb04b95ab2f4fb51e443d64 (table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d])
I20261118 02:55:20.787382  5252 tablet_loader.cc:96] loaded metadata for tablet 8bed657d0da64d95851f8a86216b6c2c (table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c])
I20261118 02:55:20.787925  5252 tablet_loader.cc:96] loaded metadata for tablet 973831392ec24e0681c3d4fffd8bb199 (table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139])
I20261118 02:55:20.789000  5252 tablet_loader.cc:96] loaded metadata for tablet a67bc60c7ecb4b7e8e03e04267bb91a5 (table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b])
I20261118 02:55:20.789693  5252 tablet_loader.cc:96] loaded metadata for tablet ab61b43e639a4e75bc2e7009ea86a64f (table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67])
I20261118 02:55:20.790086  5252 tablet_loader.cc:96] loaded metadata for tablet b129518ce23c45758ebcaa55410757d8 (table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1])
I20261118 02:55:20.790717  5252 tablet_loader.cc:96] loaded metadata for tablet b82ca8a282f94392bb2746ed7b8b8ba7 (table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d])
I20261118 02:55:20.790992  5252 tablet_loader.cc:96] loaded metadata for tablet c0713dcf79f54873a99d7556ad8f1454 (table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec])
I20261118 02:55:20.791435  5252 tablet_loader.cc:96] loaded metadata for tablet ccf78258c9634540a178855a142f4d4b (table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e])
I20261118 02:55:20.791854  5252 tablet_loader.cc:96] loaded metadata for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6])
I20261118 02:55:20.792527  5252 tablet_loader.cc:96] loaded metadata for tablet ffe8d4c111d14054b76f9c7724072a94 (table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66])
I20261118 02:55:20.792847  5252 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:55:20.794230  5252 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:20.794315  5252 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:55:20.796497  5225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 813a6937cea442ef81645fccc32207f4. Latest consensus state: current_term: 8 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:20.796767  5225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:20.797349  5252 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:55:20.797784  5254 mvcc.cc:204] Tried to move back new op lower bound from 7352199239705354240 to 7352199238828982272. Current Snapshot: MvccSnapshot[applied={T|T < 7352199239705354240}]
I20261118 02:55:20.799089  5252 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Loaded TSK: 0
I20261118 02:55:20.799180  5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:20.799484  5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:20.800175  5252 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:55:20.800987  5256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 8 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:20.801216  5256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:20.802155  5253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 8 leader_uuid: "813a6937cea442ef81645fccc32207f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:20.802385  5252 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:55:20.802462  5253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:20.803807  5252 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 20
I20261118 02:55:20.817159  4982 master_service.cc:965] ReplaceTablet: received request to replace tablet 8ae679d2feb04b95ab2f4fb51e443d64 from {username='slave'} at 127.0.0.1:37916
I20261118 02:55:20.818449  4976 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37916:
table { table_name: "default.Table_287099a68a3540f1be0284c71d4d7056" } modify_external_catalogs: true
I20261118 02:55:20.818728  4976 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37916:
table { table_name: "default.Table_287099a68a3540f1be0284c71d4d7056" } modify_external_catalogs: true
I20261118 02:55:20.819249  4975 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37916:
name: "default.Table_51d36ff687ca47b5995e5693d7b8931b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:20.820135  4981 master_service.cc:965] ReplaceTablet: received request to replace tablet c0713dcf79f54873a99d7556ad8f1454 from {username='slave'} at 127.0.0.1:37916
I20261118 02:55:20.820236  4977 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37916:
name: "default.Table_8139578347f041afbe51671d3be78447"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:20.820909  4979 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37916:
name: "default.Table_ee3815d9524243ec886bd171ccaff9e2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:20.821393  4980 master_service.cc:965] ReplaceTablet: received request to replace tablet 973831392ec24e0681c3d4fffd8bb199 from {username='slave'} at 127.0.0.1:37916
I20261118 02:55:20.822626  4983 master_service.cc:965] ReplaceTablet: received request to replace tablet ffe8d4c111d14054b76f9c7724072a94 from {username='slave'} at 127.0.0.1:37916
I20261118 02:55:20.823712  4984 master_service.cc:965] ReplaceTablet: received request to replace tablet b82ca8a282f94392bb2746ed7b8b8ba7 from {username='slave'} at 127.0.0.1:37916
I20261118 02:55:20.819316  4978 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37916:
name: "default.Table_1d3bb660e14d4101996960be8caa8f81"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
F20261118 02:55:20.831734  5036 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 15 received events: NotificationEvent(eventId=21, eventTime=1794970503, eventType=CREATE_TABLE, dbName=default, tableName=table_f0ae490a0df64e1097e144c7a370df61, message={"server":"","servicePrincipal":"","db":"default","table":"table_f0ae490a0df64e1097e144c7a370df61","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f0ae490a0df64e1097e144c7a370df61\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970503},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_f0ae490a0df64e1097e144c7a370df61\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f0ae490a0df64e1097e144c7a370df61\",\"kudu.table_id\":\"a59b9425c01049269883cf7767936c67\",\"transient_lastDdlTime\":\"1794970503\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1794970503, eventType=DROP_TABLE, dbName=default, tableName=table_c113c2b57e17479e92e241234a241af7, message={"server":"","servicePrincipal":"","db":"default","table":"table_c113c2b57e17479e92e241234a241af7","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c113c2b57e17479e92e241234a241af7\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970500},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_c113c2b57e17479e92e241234a241af7\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c113c2b57e17479e92e241234a241af7\",\"kudu.table_id\":\"da0cb25ccfed426ca61f0dd9f93d30ec\",\"transient_lastDdlTime\":\"1794970500\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
    @     0x7fcb6afd3d5d  google::LogMessage::Fail() at ??:0
    @     0x7fcb6afd7b23  google::LogMessage::SendToLog() at ??:0
    @     0x7fcb6afd375c  google::LogMessage::Flush() at ??:0
    @     0x7fcb6afd39a1  google::LogMessage::~LogMessage() at ??:0
I20261118 02:55:20.849900  3663 heartbeater.cc:500] Master 127.2.229.126:38025 was elected leader, sending a full tablet report...
I20261118 02:55:20.854040  5183 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" instance_seqno: 1794970498532688) as {username='slave'} at 127.2.229.65:43605; Asking this server to re-register.
I20261118 02:55:20.856480  3663 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:20.858747  5183 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:20.868574  4981 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Sending DeleteTablet for 3 replicas of tablet c0713dcf79f54873a99d7556ad8f1454
    @     0x7fcb804539a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 02:55:20.870682  4981 catalog_manager.cc:6936] ReplaceTablet: tablet c0713dcf79f54873a99d7556ad8f1454 deleted and replaced by tablet fbdc4c7b574e4f94b816fb0c0b14c185
    @     0x7fcb80452251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 02:55:20.880270  4980 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Sending DeleteTablet for 3 replicas of tablet 973831392ec24e0681c3d4fffd8bb199
I20261118 02:55:20.880832  4980 catalog_manager.cc:6936] ReplaceTablet: tablet 973831392ec24e0681c3d4fffd8bb199 deleted and replaced by tablet 462a9be0ae114b01a9c69416185803f8
    @     0x7fcb80459ecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 02:55:20.893810  4980 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37916:
table { table_name: "default.Table_0385cef6593c419eb426657c8cbef2bf" } new_table_name: "default.Table_496441d99faa4e2b8c7cd98e4a5ce745" modify_external_catalogs: true
I20261118 02:55:20.897856  4981 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37916:
name: "default.Table_dc982811b980416484416db1f74c685d"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:20.899350  4983 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Sending DeleteTablet for 3 replicas of tablet ffe8d4c111d14054b76f9c7724072a94
I20261118 02:55:20.899824  4983 catalog_manager.cc:6936] ReplaceTablet: tablet ffe8d4c111d14054b76f9c7724072a94 deleted and replaced by tablet b119ead4bb6548b4a37da02ed6daa6a8
I20261118 02:55:20.900300  4984 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Sending DeleteTablet for 3 replicas of tablet b82ca8a282f94392bb2746ed7b8b8ba7
I20261118 02:55:20.900713  4984 catalog_manager.cc:6936] ReplaceTablet: tablet b82ca8a282f94392bb2746ed7b8b8ba7 deleted and replaced by tablet d8766c4d56fa4689a69b153ea7a613ee
I20261118 02:55:20.901203  4982 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Sending DeleteTablet for 3 replicas of tablet 8ae679d2feb04b95ab2f4fb51e443d64
I20261118 02:55:20.901605  4982 catalog_manager.cc:6936] ReplaceTablet: tablet 8ae679d2feb04b95ab2f4fb51e443d64 deleted and replaced by tablet c46a0eff7633462fb3719143c1f7cfbb
I20261118 02:55:20.904330  4983 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37916:
table { table_name: "default.Table_8ae376d4d4d64414b5c08dc3c2e58bc4" } new_table_name: "default.Table_5091d82298ca44bea5335d4989eb02d8" modify_external_catalogs: true
I20261118 02:55:20.904771  4984 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37916:
table { table_name: "default.Table_a14cb6414555446286bf35bc5c6fb910" } new_table_name: "default.Table_324a2600fef947748ef03e27f7cb7e09" modify_external_catalogs: true
I20261118 02:55:20.905241  4982 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37916:
table { table_name: "default.Table_2e3917ff98f2474984580e61caa9c17b" } new_table_name: "default.Table_3c225da1f7fa435083909e053ec9751a" modify_external_catalogs: true
    @     0x7fcb80459d49  std::_Function_handler<>::_M_invoke() at ??:0
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
    @     0x7fcb6c41cdc9  kudu::Thread::SuperviseThread() at ??:0
    @     0x7fcb6faf76db  start_thread at ??:0
    @     0x7fcb6856761f  clone at ??:0
I20261118 02:55:20.918442  3595 tablet_service.cc:1547] Processing DeleteTablet for tablet c0713dcf79f54873a99d7556ad8f1454 with delete_type TABLET_DATA_DELETED (replaced by tablet fbdc4c7b574e4f94b816fb0c0b14c185) from {username='slave'} at 127.0.0.1:46596
I20261118 02:55:20.918504  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet 973831392ec24e0681c3d4fffd8bb199 with delete_type TABLET_DATA_DELETED (replaced by tablet 462a9be0ae114b01a9c69416185803f8) from {username='slave'} at 127.0.0.1:46596
I20261118 02:55:20.918907  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet 1d347c0ae53241708ec623562f4740d4 with delete_type TABLET_DATA_DELETED (replaced by tablet a67bc60c7ecb4b7e8e03e04267bb91a5) from {username='slave'} at 127.0.0.1:46596
I20261118 02:55:20.919237  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet ffe8d4c111d14054b76f9c7724072a94 with delete_type TABLET_DATA_DELETED (replaced by tablet b119ead4bb6548b4a37da02ed6daa6a8) from {username='slave'} at 127.0.0.1:46596
I20261118 02:55:20.919586  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet b82ca8a282f94392bb2746ed7b8b8ba7 with delete_type TABLET_DATA_DELETED (replaced by tablet d8766c4d56fa4689a69b153ea7a613ee) from {username='slave'} at 127.0.0.1:46596
I20261118 02:55:20.919940  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet 8ae679d2feb04b95ab2f4fb51e443d64 with delete_type TABLET_DATA_DELETED (replaced by tablet c46a0eff7633462fb3719143c1f7cfbb) from {username='slave'} at 127.0.0.1:46596
I20261118 02:55:20.920485  5271 tablet_replica.cc:333] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:20.920682  5271 raft_consensus.cc:2261] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:20.921073  5271 raft_consensus.cc:2290] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:20.921703  5271 ts_tablet_manager.cc:1962] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:20.930291  5271 ts_tablet_manager.cc:1975] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:20.931552  5271 log.cc:1199] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/c0713dcf79f54873a99d7556ad8f1454
I20261118 02:55:20.931989  5271 ts_tablet_manager.cc:1996] T c0713dcf79f54873a99d7556ad8f1454 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:20.933106  5271 tablet_replica.cc:333] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:20.933429  5271 raft_consensus.cc:2261] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:20.933599  5271 raft_consensus.cc:2290] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:20.934137  5271 ts_tablet_manager.cc:1962] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:20.940752  5271 ts_tablet_manager.cc:1975] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:20.940886  5271 log.cc:1199] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/973831392ec24e0681c3d4fffd8bb199
I20261118 02:55:20.941252  5271 ts_tablet_manager.cc:1996] T 973831392ec24e0681c3d4fffd8bb199 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:20.942377  5271 tablet_replica.cc:333] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:20.942574  5271 raft_consensus.cc:2261] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:20.942960  5271 raft_consensus.cc:2290] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:20.943509  5271 ts_tablet_manager.cc:1962] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:20.950045  5271 ts_tablet_manager.cc:1975] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:20.950150  5271 log.cc:1199] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/1d347c0ae53241708ec623562f4740d4
I20261118 02:55:20.950510  5271 ts_tablet_manager.cc:1996] T 1d347c0ae53241708ec623562f4740d4 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:20.951570  5271 tablet_replica.cc:333] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:20.951766  5271 raft_consensus.cc:2261] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:20.951932  5271 raft_consensus.cc:2290] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:20.952513  5271 ts_tablet_manager.cc:1962] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:20.958586  5271 ts_tablet_manager.cc:1975] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:20.958710  5271 log.cc:1199] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/ffe8d4c111d14054b76f9c7724072a94
I20261118 02:55:20.959069  5271 ts_tablet_manager.cc:1996] T ffe8d4c111d14054b76f9c7724072a94 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:20.960246  5271 tablet_replica.cc:333] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:20.960425  5271 raft_consensus.cc:2261] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:20.960587  5271 raft_consensus.cc:2290] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:20.961115  5271 ts_tablet_manager.cc:1962] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:20.964831  3800 heartbeater.cc:345] Connected to a master server at 127.2.229.126:38025
I20261118 02:55:20.967343  5183 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" instance_seqno: 1794970499028671) as {username='slave'} at 127.2.229.66:33609; Asking this server to re-register.
I20261118 02:55:20.967505  5271 ts_tablet_manager.cc:1975] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:20.967613  5271 log.cc:1199] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/b82ca8a282f94392bb2746ed7b8b8ba7
I20261118 02:55:20.968010  5271 ts_tablet_manager.cc:1996] T b82ca8a282f94392bb2746ed7b8b8ba7 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:20.968227  3800 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:20.969126  5271 tablet_replica.cc:333] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:20.969329  5271 raft_consensus.cc:2261] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:20.969493  5271 raft_consensus.cc:2290] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:20.970058  5271 ts_tablet_manager.cc:1962] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:20.971170  5183 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:20.977316  5271 ts_tablet_manager.cc:1975] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:20.977496  5271 log.cc:1199] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/8ae679d2feb04b95ab2f4fb51e443d64
I20261118 02:55:20.977895  5271 ts_tablet_manager.cc:1996] T 8ae679d2feb04b95ab2f4fb51e443d64 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
W20261118 02:55:21.110530  3956 connection.cc:573] client connection to 127.2.229.124:36695 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:21.110631  3552 connection.cc:573] server connection from 127.0.0.1:46596 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:21.111089  3956 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.229.124:36695)
W20261118 02:55:21.111009  5267 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:20.871330 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:36202 (local address 127.2.229.67:42237)
1118 02:55:20.904401 (+ 33071us) server_negotiation.cc:207] Beginning negotiation
1118 02:55:20.904408 (+     7us) server_negotiation.cc:400] Waiting for connection header
1118 02:55:20.904427 (+    19us) server_negotiation.cc:408] Connection header received
1118 02:55:20.904490 (+    63us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
1118 02:55:20.904495 (+     5us) server_negotiation.cc:462] Received NEGOTIATE request from client
1118 02:55:20.904569 (+    74us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
1118 02:55:20.906533 (+  1964us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
1118 02:55:20.906784 (+   251us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
1118 02:55:20.911023 (+  4239us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
1118 02:55:21.110959 (+199936us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36202: BlockingRecv error: recv got EOF from 127.0.0.1:36202 (error 108)
Metrics: {"server-negotiator.queue_time_us":32949,"thread_start_us":43980,"threads_started":1}
W20261118 02:55:21.111656  3956 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.229.124:36695)
W20261118 02:55:21.111814  3956 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.2.229.124:36695)
W20261118 02:55:21.111968  3956 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.2.229.124:36695)
W20261118 02:55:21.110054  3690 connection.cc:573] client connection to 127.2.229.124:36695 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:21.115104  5266 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:20.870711 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41084 (local address 127.2.229.66:40377)
1118 02:55:20.916921 (+ 46210us) server_negotiation.cc:207] Beginning negotiation
1118 02:55:20.916927 (+     6us) server_negotiation.cc:400] Waiting for connection header
1118 02:55:20.916949 (+    22us) server_negotiation.cc:408] Connection header received
1118 02:55:20.917018 (+    69us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
1118 02:55:20.917024 (+     6us) server_negotiation.cc:462] Received NEGOTIATE request from client
1118 02:55:20.917122 (+    98us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
1118 02:55:20.917229 (+   107us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
1118 02:55:21.115058 (+197829us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41084: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"server-negotiator.queue_time_us":46083,"thread_start_us":45991,"threads_started":1}
W20261118 02:55:21.117075  4790 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:21.116414 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.2.229.124:36695 (local address 127.0.0.1:37946)
1118 02:55:21.116574 (+   160us) negotiation.cc:107] Waiting for socket to connect
1118 02:55:21.116592 (+    18us) client_negotiation.cc:175] Beginning negotiation
1118 02:55:21.116705 (+   113us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 02:55:21.117013 (+   308us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":42}
W20261118 02:55:21.160531  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:21.399070  3664 heartbeater.cc:500] Master 127.2.229.124:36695 was elected leader, sending a full tablet report...
I20261118 02:55:21.411652  5282 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:21.411839  5282 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:21.413774  5282 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:21.414546  5166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:21.415915  5166 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:21.416153  5167 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:21.416280  5167 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d; no voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:21.416625  5282 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20261118 02:55:21.528079  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 1: this message will repeat every 5th retry.
W20261118 02:55:21.747941  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 1: this message will repeat every 5th retry.
W20261118 02:55:22.006570  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:22.064800  5290 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:22.064986  5290 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:22.065735  5290 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:22.067557  5166 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:22.067719  5167 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:22.067824  5167 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d; no voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:22.068064  5290 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20261118 02:55:22.742434  5299 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:22.742620  5299 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:22.743258  5299 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:22.745391  5166 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:22.745563  5167 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:22.745682  5167 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d; no voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:22.745937  5299 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20261118 02:55:22.935164  5296 raft_consensus.cc:499] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fdf7d7ece324fefb3a0c376add5b7e8)
I20261118 02:55:22.935381  5296 raft_consensus.cc:521] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:22.936177  5296 leader_election.cc:290] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:22.936852  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c0713dcf79f54873a99d7556ad8f1454" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
W20261118 02:55:22.937325  3691 leader_election.cc:343] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454
I20261118 02:55:22.937566  3690 leader_election.cc:304] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f
I20261118 02:55:22.937970  5296 raft_consensus.cc:2771] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:23.373956  5223 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:55:23.374172  5223 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:23.375036  5223 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:23.375814  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:55:23.376040  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:23.376251  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:23.376608  5223 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:23.552467  5305 raft_consensus.cc:499] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1fdf7d7ece324fefb3a0c376add5b7e8)
I20261118 02:55:23.552685  5305 raft_consensus.cc:521] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:23.553629  5305 leader_election.cc:290] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:23.554548  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c0713dcf79f54873a99d7556ad8f1454" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
W20261118 02:55:23.554819  3838 leader_election.cc:343] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454
I20261118 02:55:23.554966  3751 raft_consensus.cc:2490] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:23.555377  3836 leader_election.cc:304] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:23.555620  5305 raft_consensus.cc:2826] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20261118 02:55:23.555733  5305 raft_consensus.cc:499] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1fdf7d7ece324fefb3a0c376add5b7e8)
I20261118 02:55:23.555819  5305 raft_consensus.cc:3096] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Advancing to term 2
I20261118 02:55:23.558722  5305 raft_consensus.cc:521] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:23.559363  5305 leader_election.cc:290] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 2 election: Requested vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:23.559871  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c0713dcf79f54873a99d7556ad8f1454" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:23.560138  3751 raft_consensus.cc:3096] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Advancing to term 2
W20261118 02:55:23.560264  3838 leader_election.cc:343] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454
I20261118 02:55:23.563066  3751 raft_consensus.cc:2490] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 2.
I20261118 02:55:23.563484  3836 leader_election.cc:304] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8
I20261118 02:55:23.563917  5305 raft_consensus.cc:2826] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 2 FOLLOWER]: Leader election won for term 2
I20261118 02:55:23.564332  5305 raft_consensus.cc:705] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [term 2 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:23.564640  5305 consensus_queue.cc:238] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:23.566792  3751 raft_consensus.cc:1285] T c0713dcf79f54873a99d7556ad8f1454 P 99a39b1d9d814deeb3a8a644e9b392ef [term 2 FOLLOWER]: Refusing update from remote peer 3f1973986ec1466eab5bec01dd81654f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20261118 02:55:23.567268  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:23.567396  5305 consensus_queue.cc:1049] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20261118 02:55:23.589186  5223 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:55:23.589414  5223 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:23.590181  5223 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:23.590929  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:55:23.591190  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:23.591316  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:23.591562  5223 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:23.960971  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.125:38339
--webserver_interface=127.2.229.125
--webserver_port=39093
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 02:55:23.990309  5316 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:23.990501  5316 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:23.991230  5316 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:23.993335  5166 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:23.993484  5167 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:23.993597  5167 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d; no voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:23.993933  5316 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20261118 02:55:24.303120  5315 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:55:24.303491  5315 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:55:24.303613  5315 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:55:24.313185  5315 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:55:24.313288  5315 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:55:24.313366  5315 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:55:24.313433  5315 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:55:24.323764  5315 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.125:38339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.229.125
--webserver_port=39093
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:55:24.325577  5315 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:55:24.327409  5315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:55:24.335559  5331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:24.335638  5330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:24.336400  5333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:24.336812  5315 server_base.cc:1061] running on GCE node
I20261118 02:55:24.337532  5315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:55:24.339097  5315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:55:24.340365  5315 hybrid_clock.cc:648] HybridClock initialized: now 1794970524340321 us; error 41 us; skew 500 ppm
I20261118 02:55:24.340730  5315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:55:24.343422  5315 webserver.cc:577] Webserver started at http://127.2.229.125:39093/ using document root <none> and password file <none>
I20261118 02:55:24.344174  5315 fs_manager.cc:362] Metadata directory not provided
I20261118 02:55:24.344302  5315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:55:24.350091  5315 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20261118 02:55:24.353492  5339 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:55:24.355297  5315 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20261118 02:55:24.355468  5315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
uuid: "457eb38cba564ecd9f6f11245172260b"
format_stamp: "Formatted at 2026-11-18 02:54:56 on dist-test-slave-49p1"
I20261118 02:55:24.356257  5315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:55:24.380457  5315 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:55:24.381372  5315 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:55:24.381637  5315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:55:24.399927  5315 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.125:38339
I20261118 02:55:24.399945  5390 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.125:38339 every 8 connection(s)
I20261118 02:55:24.401556  5315 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-1/data/info.pb
I20261118 02:55:24.408874  5391 sys_catalog.cc:263] Verifying existing consensus state
I20261118 02:55:24.409404  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 5315
I20261118 02:55:24.413405  5391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap starting.
I20261118 02:55:24.435981  5391 log.cc:826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:24.511309  5391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap replayed 1/1 log segments. Stats: ops{read=106 overwritten=0 applied=106 ignored=0} inserts{seen=70 ignored=0} mutations{seen=93 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:24.511799  5391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Bootstrap complete.
I20261118 02:55:24.517663  5391 raft_consensus.cc:360] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:24.518399  5391 raft_consensus.cc:748] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Initialized, Role: FOLLOWER
I20261118 02:55:24.519016  5391 consensus_queue.cc:261] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 106, Last appended: 7.106, Last appended by leader: 106, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:24.519794  5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:24.520058  5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:24.520825  5391 sys_catalog.cc:565] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:55:24.568730  5410 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:24.568949  5410 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: loading cluster ID for follower catalog manager: success
I20261118 02:55:24.574469  5410 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: acquiring CA information for follower catalog manager: success
I20261118 02:55:24.581694  5410 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:55:24.576993  5315 master_runner.cc:187] Error getting master registration for 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111), 
I20261118 02:55:24.582908  5315 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:24.651752  3949 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:24.656266  5356 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" instance_seqno: 1794970499527172) as {username='slave'} at 127.2.229.67:38051; Asking this server to re-register.
I20261118 02:55:24.657300  3949 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:24.659678  5356 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:24.765259  3802 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:24.768347  5356 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" instance_seqno: 1794970499028671) as {username='slave'} at 127.2.229.66:45571; Asking this server to re-register.
I20261118 02:55:24.769181  3802 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:24.771147  5356 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:24.871848  3665 heartbeater.cc:345] Connected to a master server at 127.2.229.125:38339
I20261118 02:55:24.873872  5356 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" instance_seqno: 1794970498532688) as {username='slave'} at 127.2.229.65:56813; Asking this server to re-register.
I20261118 02:55:24.874491  3665 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:24.875998  5356 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:25.036291  5421 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:25.036616  5421 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:25.038511  5421 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:25.039032  5193 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 8 candidate_status { last_received { term: 7 index: 106 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
W20261118 02:55:25.039248  5342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:25.039367  5193 raft_consensus.cc:2415] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 457eb38cba564ecd9f6f11245172260b in current term 8: Already voted for candidate 813a6937cea442ef81645fccc32207f4 in this term.
W20261118 02:55:25.040669  5342 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:25.040836  5342 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:25.041169  5421 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 7 FOLLOWER]: Advancing to term 8
I20261118 02:55:25.045043  5421 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20261118 02:55:25.565763  5426 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:25.565958  5426 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:25.566581  5426 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:25.567063  5193 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 9 candidate_status { last_received { term: 7 index: 106 } } ignore_live_leader: false dest_uuid: "26c6b20fa203404a88463ea27bdd3c7d" is_pre_election: true
I20261118 02:55:25.567327  5193 raft_consensus.cc:2432] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 457eb38cba564ecd9f6f11245172260b for term 9 because replica has last-logged OpId of term: 8 index: 113, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 106.
W20261118 02:55:25.568538  5342 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:25.568653  5342 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:25.568904  5426 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20261118 02:55:25.699878  5146 master_runner.cc:187] Error getting master registration for 127.2.229.124:36695: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111), 
I20261118 02:55:25.700084  5146 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:26.314917  5436 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
I20261118 02:55:26.315114  5436 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:26.315905  5436 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:26.316627  5366 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 9 candidate_status { last_received { term: 8 index: 113 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b" is_pre_election: true
I20261118 02:55:26.316994  5366 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 8.
I20261118 02:55:26.317548  5166 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b; no voters: 
I20261118 02:55:26.317836  5436 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Leader pre-election won for term 9
I20261118 02:55:26.317984  5436 raft_consensus.cc:499] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Starting leader election (detected failure of leader 813a6937cea442ef81645fccc32207f4)
W20261118 02:55:26.318037  5167 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:26.318125  5436 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 8 FOLLOWER]: Advancing to term 9
I20261118 02:55:26.320196  5436 raft_consensus.cc:521] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:26.320786  5436 leader_election.cc:290] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 election: Requested vote from peers 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:26.321272  5366 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "26c6b20fa203404a88463ea27bdd3c7d" candidate_term: 9 candidate_status { last_received { term: 8 index: 113 } } ignore_live_leader: false dest_uuid: "457eb38cba564ecd9f6f11245172260b"
I20261118 02:55:26.321465  5366 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 8 FOLLOWER]: Advancing to term 9
W20261118 02:55:26.322074  5167 leader_election.cc:336] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:26.323616  5366 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c6b20fa203404a88463ea27bdd3c7d in term 9.
I20261118 02:55:26.324146  5166 leader_election.cc:304] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b; no voters: 813a6937cea442ef81645fccc32207f4
I20261118 02:55:26.324491  5436 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 9 FOLLOWER]: Leader election won for term 9
I20261118 02:55:26.324888  5436 raft_consensus.cc:705] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [term 9 LEADER]: Becoming Leader. State: Replica: 26c6b20fa203404a88463ea27bdd3c7d, State: Running, Role: LEADER
I20261118 02:55:26.325345  5436 consensus_queue.cc:238] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 113, Committed index: 113, Last appended: 8.113, Last appended by leader: 113, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:26.326917  5439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 9 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:26.327276  5439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:26.328112  5443 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:55:26.329102  5366 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Refusing update from remote peer 26c6b20fa203404a88463ea27bdd3c7d: Log matching property violated. Preceding OpId in replica: term: 7 index: 106. Preceding OpId from leader: term: 9 index: 114. (index mismatch)
W20261118 02:55:26.329769  5167 consensus_peers.cc:597] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d -> Peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Couldn't send request to peer 813a6937cea442ef81645fccc32207f4. Status: Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:26.330085  5441 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [LEADER]: Connected to new peer: Peer: permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 114, Last known committed idx: 106, Time since last communication: 0.000s
I20261118 02:55:26.331959  5443 catalog_manager.cc:694] Loaded metadata for table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]
I20261118 02:55:26.332523  5443 catalog_manager.cc:694] Loaded metadata for table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]
I20261118 02:55:26.332958  5443 catalog_manager.cc:694] Loaded metadata for table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]
I20261118 02:55:26.333416  5443 catalog_manager.cc:694] Loaded metadata for table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]
I20261118 02:55:26.333820  5443 catalog_manager.cc:694] Loaded metadata for table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]
I20261118 02:55:26.334268  5443 catalog_manager.cc:694] Loaded metadata for table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]
I20261118 02:55:26.334888  5443 catalog_manager.cc:694] Loaded metadata for table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da]
I20261118 02:55:26.335407  5443 catalog_manager.cc:694] Loaded metadata for table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
I20261118 02:55:26.335824  5443 catalog_manager.cc:694] Loaded metadata for table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]
I20261118 02:55:26.336251  5443 catalog_manager.cc:694] Loaded metadata for table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:26.336658  5443 catalog_manager.cc:694] Loaded metadata for table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:26.337522  5443 catalog_manager.cc:694] Loaded metadata for table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b]
I20261118 02:55:26.337939  5443 catalog_manager.cc:694] Loaded metadata for table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]
I20261118 02:55:26.338344  5443 catalog_manager.cc:694] Loaded metadata for table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]
I20261118 02:55:26.338749  5443 catalog_manager.cc:694] Loaded metadata for table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]
I20261118 02:55:26.339143  5443 catalog_manager.cc:694] Loaded metadata for table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]
I20261118 02:55:26.339550  5443 catalog_manager.cc:694] Loaded metadata for table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:26.339964  5443 catalog_manager.cc:694] Loaded metadata for table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]
I20261118 02:55:26.340418  5443 catalog_manager.cc:694] Loaded metadata for table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]
I20261118 02:55:26.340855  5443 catalog_manager.cc:694] Loaded metadata for table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]
I20261118 02:55:26.344050  5443 tablet_loader.cc:96] loaded metadata for tablet 01778879aaa44ea9855fae67cfb3952d (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7])
I20261118 02:55:26.344540  5443 tablet_loader.cc:96] loaded metadata for tablet 0e901a03b90e4c978eb98be87163cfe5 (table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190])
I20261118 02:55:26.344913  5443 tablet_loader.cc:96] loaded metadata for tablet 1904140da78b4567a9decb250c6f74a3 (table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f])
I20261118 02:55:26.345512  5443 tablet_loader.cc:96] loaded metadata for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db])
I20261118 02:55:26.345849  5443 tablet_loader.cc:96] loaded metadata for tablet 2c92955aa65142f1bd20f4b8f3321500 (table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d])
I20261118 02:55:26.346398  5443 tablet_loader.cc:96] loaded metadata for tablet 462a9be0ae114b01a9c69416185803f8 (table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139])
I20261118 02:55:26.346750  5443 tablet_loader.cc:96] loaded metadata for tablet 46f05790dbdc4b8185a7b0362f604cf8 (table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf])
I20261118 02:55:26.348302  5443 tablet_loader.cc:96] loaded metadata for tablet 65bd6fcaed5b47c3a2478d375ace9636 (table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73])
I20261118 02:55:26.348794  5445 mvcc.cc:204] Tried to move back new op lower bound from 7352199253305876480 to 7352199253043658752. Current Snapshot: MvccSnapshot[applied={T|T < 7352199253305876480}]
I20261118 02:55:26.348979  5443 tablet_loader.cc:96] loaded metadata for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b])
I20261118 02:55:26.349395  5443 tablet_loader.cc:96] loaded metadata for tablet 78189cccd457466bb5d01231038fec24 (table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da])
I20261118 02:55:26.349987  5443 tablet_loader.cc:96] loaded metadata for tablet 8bed657d0da64d95851f8a86216b6c2c (table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c])
I20261118 02:55:26.351516  5443 tablet_loader.cc:96] loaded metadata for tablet a67bc60c7ecb4b7e8e03e04267bb91a5 (table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b])
I20261118 02:55:26.352082  5443 tablet_loader.cc:96] loaded metadata for tablet ab61b43e639a4e75bc2e7009ea86a64f (table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67])
I20261118 02:55:26.352458  5443 tablet_loader.cc:96] loaded metadata for tablet b119ead4bb6548b4a37da02ed6daa6a8 (table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66])
I20261118 02:55:26.352864  5443 tablet_loader.cc:96] loaded metadata for tablet b129518ce23c45758ebcaa55410757d8 (table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1])
I20261118 02:55:26.353163  5444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c6b20fa203404a88463ea27bdd3c7d. Latest consensus state: current_term: 9 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:26.353422  5444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:26.353843  5443 tablet_loader.cc:96] loaded metadata for tablet c46a0eff7633462fb3719143c1f7cfbb (table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d])
I20261118 02:55:26.354539  5443 tablet_loader.cc:96] loaded metadata for tablet ccf78258c9634540a178855a142f4d4b (table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e])
I20261118 02:55:26.354481  5447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:26.354820  5447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:26.355557  5443 tablet_loader.cc:96] loaded metadata for tablet d8766c4d56fa4689a69b153ea7a613ee (table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d])
I20261118 02:55:26.355520  5447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 9 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:26.355824  5447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:26.356273  5443 tablet_loader.cc:96] loaded metadata for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6])
I20261118 02:55:26.356190  5442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 9 leader_uuid: "26c6b20fa203404a88463ea27bdd3c7d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:26.356467  5442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:26.357071  5443 tablet_loader.cc:96] loaded metadata for tablet fbdc4c7b574e4f94b816fb0c0b14c185 (table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec])
I20261118 02:55:26.357542  5443 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:55:26.358835  5443 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:26.358901  5443 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:55:26.360419  5443 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:55:26.361577  5443 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Loaded TSK: 0
I20261118 02:55:26.362411  5443 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:55:26.363606  5443 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:55:26.364877  5443 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 20
I20261118 02:55:26.377704  5182 master_service.cc:965] ReplaceTablet: received request to replace tablet 01778879aaa44ea9855fae67cfb3952d from {username='slave'} at 127.0.0.1:59696
I20261118 02:55:26.377913  5183 master_service.cc:965] ReplaceTablet: received request to replace tablet 65bd6fcaed5b47c3a2478d375ace9636 from {username='slave'} at 127.0.0.1:59696
I20261118 02:55:26.378772  5177 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59696:
table { table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" } modify_external_catalogs: true
I20261118 02:55:26.379175  5179 master_service.cc:965] ReplaceTablet: received request to replace tablet 8ae679d2feb04b95ab2f4fb51e443d64 from {username='slave'} at 127.0.0.1:59696
I20261118 02:55:26.379484  5181 master_service.cc:965] ReplaceTablet: received request to replace tablet ffe8d4c111d14054b76f9c7724072a94 from {username='slave'} at 127.0.0.1:59696
I20261118 02:55:26.380021  5179 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59696:
table { table_name: "default.Table_571d1360811c42dc815efb439b2b52e6" } modify_external_catalogs: true
I20261118 02:55:26.380180  5180 master_service.cc:965] ReplaceTablet: received request to replace tablet 973831392ec24e0681c3d4fffd8bb199 from {username='slave'} at 127.0.0.1:59696
I20261118 02:55:26.380275  5179 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59696:
table { table_name: "default.Table_571d1360811c42dc815efb439b2b52e6" } modify_external_catalogs: true
I20261118 02:55:26.380223  5174 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59696:
table { table_name: "default.Table_28c62f411da74545aa42dad7325bc064" } modify_external_catalogs: true
I20261118 02:55:26.380674  5174 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59696:
table { table_name: "default.Table_28c62f411da74545aa42dad7325bc064" } modify_external_catalogs: true
I20261118 02:55:26.380440  5176 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59696:
table { table_name: "default.Table_4f7796d5d5d54def843b971e66d15ea6" } new_table_name: "default.Table_79eb11797e3f4bc2bffe161096300f7a" modify_external_catalogs: true
I20261118 02:55:26.378835  5178 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59696:
name: "default.Table_fd4fcac1f1d946fa898176f1d5e87e9b"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:26.380759  5177 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59696:
table { table_name: "default.Table_a7f3710d39fb447da94e16d4ab3933fd" } modify_external_catalogs: true
I20261118 02:55:26.379794  5175 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59696:
name: "default.Table_e39f59e2fdbb47fda4c10beba29ad8f1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:26.383098  5180 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59696:
name: "default.Table_52fca7a2e65f4e679bbef9de3df569f1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:26.383123  5181 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59696:
name: "default.Table_40868189cecf45d0b705461c4328ee17"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
F20261118 02:55:26.390275  5238 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 15 received events: NotificationEvent(eventId=21, eventTime=1794970503, eventType=CREATE_TABLE, dbName=default, tableName=table_f0ae490a0df64e1097e144c7a370df61, message={"server":"","servicePrincipal":"","db":"default","table":"table_f0ae490a0df64e1097e144c7a370df61","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f0ae490a0df64e1097e144c7a370df61\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970503},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_f0ae490a0df64e1097e144c7a370df61\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f0ae490a0df64e1097e144c7a370df61\",\"kudu.table_id\":\"a59b9425c01049269883cf7767936c67\",\"transient_lastDdlTime\":\"1794970503\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1794970503, eventType=DROP_TABLE, dbName=default, tableName=table_c113c2b57e17479e92e241234a241af7, message={"server":"","servicePrincipal":"","db":"default","table":"table_c113c2b57e17479e92e241234a241af7","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c113c2b57e17479e92e241234a241af7\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970500},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_c113c2b57e17479e92e241234a241af7\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c113c2b57e17479e92e241234a241af7\",\"kudu.table_id\":\"da0cb25ccfed426ca61f0dd9f93d30ec\",\"transient_lastDdlTime\":\"1794970500\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
    @     0x7f55745a2d5d  google::LogMessage::Fail() at ??:0
    @     0x7f55745a6b23  google::LogMessage::SendToLog() at ??:0
I20261118 02:55:26.395017  5182 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet 01778879aaa44ea9855fae67cfb3952d
    @     0x7f55745a275c  google::LogMessage::Flush() at ??:0
I20261118 02:55:26.397074  5182 catalog_manager.cc:6936] ReplaceTablet: tablet 01778879aaa44ea9855fae67cfb3952d deleted and replaced by tablet b400925bbe6f4639b3d9e76cbb1ca681
I20261118 02:55:26.399232  5183 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 26c6b20fa203404a88463ea27bdd3c7d: Sending DeleteTablet for 3 replicas of tablet 65bd6fcaed5b47c3a2478d375ace9636
I20261118 02:55:26.399708  5183 catalog_manager.cc:6936] ReplaceTablet: tablet 65bd6fcaed5b47c3a2478d375ace9636 deleted and replaced by tablet c1f0147a1efb4f4f813478f119043bec
    @     0x7f55745a29a1  google::LogMessage::~LogMessage() at ??:0
I20261118 02:55:26.422303  5182 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59696:
name: "default.Table_ee3815d9524243ec886bd171ccaff9e2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:26.422988  5183 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59696:
name: "default.Table_bdbed231860946c2bf9b514f724bc3e9"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
    @     0x7f5589a229a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 02:55:26.440920  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet 01778879aaa44ea9855fae67cfb3952d with delete_type TABLET_DATA_DELETED (replaced by tablet b400925bbe6f4639b3d9e76cbb1ca681) from {username='slave'} at 127.0.0.1:35078
I20261118 02:55:26.441577  5461 tablet_replica.cc:333] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:26.441788  5461 raft_consensus.cc:2261] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:26.441967  3732 tablet_service.cc:1547] Processing DeleteTablet for tablet 01778879aaa44ea9855fae67cfb3952d with delete_type TABLET_DATA_DELETED (replaced by tablet b400925bbe6f4639b3d9e76cbb1ca681) from {username='slave'} at 127.0.0.1:50108
I20261118 02:55:26.442023  3881 tablet_service.cc:1547] Processing DeleteTablet for tablet 65bd6fcaed5b47c3a2478d375ace9636 with delete_type TABLET_DATA_DELETED (replaced by tablet c1f0147a1efb4f4f813478f119043bec) from {username='slave'} at 127.0.0.1:35078
I20261118 02:55:26.442066  5461 raft_consensus.cc:2290] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:26.442253  3733 tablet_service.cc:1547] Processing DeleteTablet for tablet 65bd6fcaed5b47c3a2478d375ace9636 with delete_type TABLET_DATA_DELETED (replaced by tablet c1f0147a1efb4f4f813478f119043bec) from {username='slave'} at 127.0.0.1:50108
I20261118 02:55:26.442514  3734 tablet_service.cc:1500] Processing CreateTablet for tablet c46a0eff7633462fb3719143c1f7cfbb (DEFAULT_TABLE table=default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.442624  3735 tablet_service.cc:1500] Processing CreateTablet for tablet c1f0147a1efb4f4f813478f119043bec (DEFAULT_TABLE table=default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.442960  3734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46a0eff7633462fb3719143c1f7cfbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.443212  3733 tablet_service.cc:1500] Processing CreateTablet for tablet b119ead4bb6548b4a37da02ed6daa6a8 (DEFAULT_TABLE table=default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.443612  3733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b119ead4bb6548b4a37da02ed6daa6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.443516  3731 tablet_service.cc:1500] Processing CreateTablet for tablet d8766c4d56fa4689a69b153ea7a613ee (DEFAULT_TABLE table=default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.444670  3731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8766c4d56fa4689a69b153ea7a613ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.445508  3735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f0147a1efb4f4f813478f119043bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.449819  5461 ts_tablet_manager.cc:1962] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:26.449874  3881 tablet_service.cc:1500] Processing CreateTablet for tablet c46a0eff7633462fb3719143c1f7cfbb (DEFAULT_TABLE table=default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.450317  3881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46a0eff7633462fb3719143c1f7cfbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.452347  5462 tablet_replica.cc:333] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:26.453279  3877 tablet_service.cc:1500] Processing CreateTablet for tablet c1f0147a1efb4f4f813478f119043bec (DEFAULT_TABLE table=default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.453666  3877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f0147a1efb4f4f813478f119043bec. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.453908  3882 tablet_service.cc:1500] Processing CreateTablet for tablet b400925bbe6f4639b3d9e76cbb1ca681 (DEFAULT_TABLE table=default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.453910  3878 tablet_service.cc:1500] Processing CreateTablet for tablet d8766c4d56fa4689a69b153ea7a613ee (DEFAULT_TABLE table=default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.454357  3882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b400925bbe6f4639b3d9e76cbb1ca681. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.453676  3880 tablet_service.cc:1500] Processing CreateTablet for tablet b119ead4bb6548b4a37da02ed6daa6a8 (DEFAULT_TABLE table=default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.459160  3878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8766c4d56fa4689a69b153ea7a613ee. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f5589a21251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
I20261118 02:55:26.465845  3595 tablet_service.cc:1547] Processing DeleteTablet for tablet 01778879aaa44ea9855fae67cfb3952d with delete_type TABLET_DATA_DELETED (replaced by tablet b400925bbe6f4639b3d9e76cbb1ca681) from {username='slave'} at 127.0.0.1:46360
I20261118 02:55:26.465983  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet 65bd6fcaed5b47c3a2478d375ace9636 with delete_type TABLET_DATA_DELETED (replaced by tablet c1f0147a1efb4f4f813478f119043bec) from {username='slave'} at 127.0.0.1:46360
I20261118 02:55:26.466496  3597 tablet_service.cc:1500] Processing CreateTablet for tablet b119ead4bb6548b4a37da02ed6daa6a8 (DEFAULT_TABLE table=default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.466737  3880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b119ead4bb6548b4a37da02ed6daa6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.466688  3594 tablet_service.cc:1500] Processing CreateTablet for tablet c46a0eff7633462fb3719143c1f7cfbb (DEFAULT_TABLE table=default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.466881  3597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b119ead4bb6548b4a37da02ed6daa6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.467092  3594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c46a0eff7633462fb3719143c1f7cfbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.472448  3598 tablet_service.cc:1500] Processing CreateTablet for tablet c1f0147a1efb4f4f813478f119043bec (DEFAULT_TABLE table=default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.472841  3598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1f0147a1efb4f4f813478f119043bec. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f5589a28ecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20261118 02:55:26.476421  3596 tablet_service.cc:1500] Processing CreateTablet for tablet d8766c4d56fa4689a69b153ea7a613ee (DEFAULT_TABLE table=default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.476763  3596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8766c4d56fa4689a69b153ea7a613ee. 1 dirs total, 0 dirs full, 0 dirs failed
    @     0x7f5589a28d49  std::_Function_handler<>::_M_invoke() at ??:0
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
I20261118 02:55:26.480116  3592 tablet_service.cc:1500] Processing CreateTablet for tablet b400925bbe6f4639b3d9e76cbb1ca681 (DEFAULT_TABLE table=default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.480607  3592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b400925bbe6f4639b3d9e76cbb1ca681. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.453014  3730 tablet_service.cc:1500] Processing CreateTablet for tablet b400925bbe6f4639b3d9e76cbb1ca681 (DEFAULT_TABLE table=default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20261118 02:55:26.481835  3730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b400925bbe6f4639b3d9e76cbb1ca681. 1 dirs total, 0 dirs full, 0 dirs failed
I20261118 02:55:26.483476  5462 raft_consensus.cc:2261] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
    @     0x7f55759ebdc9  kudu::Thread::SuperviseThread() at ??:0
    @     0x7f55790c66db  start_thread at ??:0
    @     0x7f5571b3661f  clone at ??:0
I20261118 02:55:26.497404  5462 raft_consensus.cc:2290] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:26.498037  5462 ts_tablet_manager.cc:1962] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20261118 02:55:26.502343  3838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.229.65:33705, user_credentials={real_user=slave}} blocked reactor thread for 47933.7us
I20261118 02:55:26.504607  5470 tablet_bootstrap.cc:492] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:26.507344  5469 tablet_bootstrap.cc:492] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:26.510207  5469 tablet_bootstrap.cc:654] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.511461  5467 tablet_replica.cc:333] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:26.511698  5467 raft_consensus.cc:2261] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:26.512156  5467 raft_consensus.cc:2290] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:26.512800  5467 ts_tablet_manager.cc:1962] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:26.513556  5461 ts_tablet_manager.cc:1975] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:26.513674  5461 log.cc:1199] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/01778879aaa44ea9855fae67cfb3952d
I20261118 02:55:26.514061  5461 ts_tablet_manager.cc:1996] T 01778879aaa44ea9855fae67cfb3952d P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:26.515072  5461 tablet_replica.cc:333] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: stopping tablet replica
I20261118 02:55:26.515334  5461 raft_consensus.cc:2261] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Raft consensus shutting down.
I20261118 02:55:26.515669  5461 raft_consensus.cc:2290] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:26.516408  5470 tablet_bootstrap.cc:654] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.517941  5468 tablet_bootstrap.cc:492] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:26.519225  5462 ts_tablet_manager.cc:1975] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:26.519313  5462 log.cc:1199] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/01778879aaa44ea9855fae67cfb3952d
I20261118 02:55:26.519587  5462 ts_tablet_manager.cc:1996] T 01778879aaa44ea9855fae67cfb3952d P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:26.519951  5468 tablet_bootstrap.cc:654] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.520807  5462 tablet_replica.cc:333] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: stopping tablet replica
I20261118 02:55:26.521095  5462 raft_consensus.cc:2261] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:26.521545  5462 raft_consensus.cc:2290] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:26.521744  5461 ts_tablet_manager.cc:1962] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:26.522405  5470 tablet_bootstrap.cc:492] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:26.522540  5470 ts_tablet_manager.cc:1449] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.018s	user 0.002s	sys 0.002s
I20261118 02:55:26.523609  5470 raft_consensus.cc:360] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.523804  5470 raft_consensus.cc:386] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.523883  5470 raft_consensus.cc:748] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.527484  5462 ts_tablet_manager.cc:1962] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:26.528033  5467 ts_tablet_manager.cc:1975] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20261118 02:55:26.528185  5467 log.cc:1199] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/01778879aaa44ea9855fae67cfb3952d
I20261118 02:55:26.528590  5467 ts_tablet_manager.cc:1996] T 01778879aaa44ea9855fae67cfb3952d P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:26.528873  5461 ts_tablet_manager.cc:1975] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:26.529016  5461 log.cc:1199] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-2/wal/wals/65bd6fcaed5b47c3a2478d375ace9636
I20261118 02:55:26.529417  5461 ts_tablet_manager.cc:1996] T 65bd6fcaed5b47c3a2478d375ace9636 P 3f1973986ec1466eab5bec01dd81654f: Deleting consensus metadata
I20261118 02:55:26.529667  5467 tablet_replica.cc:333] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:26.529891  5467 raft_consensus.cc:2261] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:26.530104  5467 raft_consensus.cc:2290] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:26.531332  5469 tablet_bootstrap.cc:492] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:26.531466  5469 ts_tablet_manager.cc:1449] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.024s	user 0.000s	sys 0.004s
I20261118 02:55:26.532280  5469 raft_consensus.cc:360] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.532459  5469 raft_consensus.cc:386] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.532531  5469 raft_consensus.cc:748] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.532830  5470 consensus_queue.cc:261] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.536119  5470 ts_tablet_manager.cc:1480] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.013s	user 0.000s	sys 0.002s
I20261118 02:55:26.537688  5469 consensus_queue.cc:261] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.540684  5462 ts_tablet_manager.cc:1975] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:26.540812  5462 log.cc:1199] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-1/wal/wals/65bd6fcaed5b47c3a2478d375ace9636
I20261118 02:55:26.541198  5462 ts_tablet_manager.cc:1996] T 65bd6fcaed5b47c3a2478d375ace9636 P 99a39b1d9d814deeb3a8a644e9b392ef: Deleting consensus metadata
I20261118 02:55:26.552300  5470 tablet_bootstrap.cc:492] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:26.554435  5470 tablet_bootstrap.cc:654] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.558333  5469 ts_tablet_manager.cc:1480] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.027s	user 0.002s	sys 0.000s
I20261118 02:55:26.559616  5469 tablet_bootstrap.cc:492] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:26.562727  5469 tablet_bootstrap.cc:654] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.564500  5468 tablet_bootstrap.cc:492] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:26.564653  5468 ts_tablet_manager.cc:1449] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.047s	user 0.007s	sys 0.013s
I20261118 02:55:26.565452  5468 raft_consensus.cc:360] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.565642  5468 raft_consensus.cc:386] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.565723  5468 raft_consensus.cc:748] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.565989  5468 consensus_queue.cc:261] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.566454  3663 heartbeater.cc:500] Master 127.2.229.126:38025 was elected leader, sending a full tablet report...
I20261118 02:55:26.568331  5468 ts_tablet_manager.cc:1480] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 02:55:26.571219  5469 tablet_bootstrap.cc:492] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:26.571380  5469 ts_tablet_manager.cc:1449] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.012s	user 0.005s	sys 0.004s
I20261118 02:55:26.572203  5469 raft_consensus.cc:360] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.572392  5469 raft_consensus.cc:386] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.572455  5469 raft_consensus.cc:748] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.572738  5469 consensus_queue.cc:261] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.573143  5470 tablet_bootstrap.cc:492] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:26.573309  5470 ts_tablet_manager.cc:1449] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.037s	user 0.000s	sys 0.003s
I20261118 02:55:26.575390  5469 ts_tablet_manager.cc:1480] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 02:55:26.574066  5470 raft_consensus.cc:360] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.576234  5470 raft_consensus.cc:386] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.576318  5470 raft_consensus.cc:748] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.576576  5470 consensus_queue.cc:261] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.577075  5470 ts_tablet_manager.cc:1480] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.000s
I20261118 02:55:26.577852  5469 tablet_bootstrap.cc:492] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:26.580029  5469 tablet_bootstrap.cc:654] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.580224  5223 raft_consensus.cc:499] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:26.580389  5223 raft_consensus.cc:521] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.581095  5223 leader_election.cc:290] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.581457  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b119ead4bb6548b4a37da02ed6daa6a8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:26.581952  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b119ead4bb6548b4a37da02ed6daa6a8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:26.582132  5470 tablet_bootstrap.cc:492] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:26.582348  3751 raft_consensus.cc:2490] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
W20261118 02:55:26.582592  3553 leader_election.cc:343] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Illegal state: must be running to vote when last-logged opid is not known
I20261118 02:55:26.583514  3552 leader_election.cc:304] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 3f1973986ec1466eab5bec01dd81654f
I20261118 02:55:26.584192  5470 tablet_bootstrap.cc:654] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.585340  5223 raft_consensus.cc:2826] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:26.585444  5223 raft_consensus.cc:499] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:26.585517  5223 raft_consensus.cc:3096] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.588392  5223 raft_consensus.cc:521] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.590905  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b119ead4bb6548b4a37da02ed6daa6a8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:26.591154  5468 tablet_bootstrap.cc:492] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
W20261118 02:55:26.592422  3553 leader_election.cc:343] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Illegal state: must be running to vote when last-logged opid is not known
I20261118 02:55:26.592964  5468 tablet_bootstrap.cc:654] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.595716  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b119ead4bb6548b4a37da02ed6daa6a8" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:26.595944  3751 raft_consensus.cc:3096] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.597872  5468 tablet_bootstrap.cc:492] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:26.598038  5468 ts_tablet_manager.cc:1449] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.007s	user 0.000s	sys 0.003s
I20261118 02:55:26.598819  5468 raft_consensus.cc:360] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.599017  5468 raft_consensus.cc:386] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.599088  5468 raft_consensus.cc:748] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.599347  5468 consensus_queue.cc:261] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.599607  3751 raft_consensus.cc:2490] T b119ead4bb6548b4a37da02ed6daa6a8 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:26.600179  5223 leader_election.cc:290] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.600370  3552 leader_election.cc:304] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 3f1973986ec1466eab5bec01dd81654f
I20261118 02:55:26.601939  5468 ts_tablet_manager.cc:1480] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.001s
I20261118 02:55:26.602519  5467 ts_tablet_manager.cc:1962] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:26.603873  5223 raft_consensus.cc:2826] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:26.604066  5223 raft_consensus.cc:705] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:26.604453  5223 consensus_queue.cc:238] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.603153  5468 tablet_bootstrap.cc:492] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:26.609411  5471 consensus_queue.cc:1049] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:26.611416  5468 tablet_bootstrap.cc:654] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
W20261118 02:55:26.611650  3553 consensus_peers.cc:597] T b119ead4bb6548b4a37da02ed6daa6a8 P 1fdf7d7ece324fefb3a0c376add5b7e8 -> Peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Couldn't send request to peer 3f1973986ec1466eab5bec01dd81654f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:26.619196  5467 ts_tablet_manager.cc:1975] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:26.619350  5467 log.cc:1199] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/65bd6fcaed5b47c3a2478d375ace9636
I20261118 02:55:26.622983  5469 tablet_bootstrap.cc:492] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:26.623145  5469 ts_tablet_manager.cc:1449] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.045s	user 0.003s	sys 0.000s
I20261118 02:55:26.623934  5469 raft_consensus.cc:360] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.624138  5469 raft_consensus.cc:386] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.624231  5469 raft_consensus.cc:748] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.624473  5469 consensus_queue.cc:261] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.625037  5469 ts_tablet_manager.cc:1480] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:26.625331  5469 tablet_bootstrap.cc:492] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:26.627117  5469 tablet_bootstrap.cc:654] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.628249  5467 ts_tablet_manager.cc:1996] T 65bd6fcaed5b47c3a2478d375ace9636 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:26.630618  5468 tablet_bootstrap.cc:492] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:26.630762  5468 ts_tablet_manager.cc:1449] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.028s	user 0.003s	sys 0.003s
I20261118 02:55:26.631496  5468 raft_consensus.cc:360] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.631680  5468 raft_consensus.cc:386] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.631759  5468 raft_consensus.cc:748] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.632010  5468 consensus_queue.cc:261] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.632588  5468 ts_tablet_manager.cc:1480] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:26.635655  5468 tablet_bootstrap.cc:492] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:26.637357  5468 tablet_bootstrap.cc:654] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.649585  5470 tablet_bootstrap.cc:492] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:26.649569  5469 tablet_bootstrap.cc:492] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:26.649730  5470 ts_tablet_manager.cc:1449] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.025s
I20261118 02:55:26.649747  5469 ts_tablet_manager.cc:1449] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.005s
I20261118 02:55:26.650496  5469 raft_consensus.cc:360] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.650521  5470 raft_consensus.cc:360] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.650682  5470 raft_consensus.cc:386] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.650699  5469 raft_consensus.cc:386] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.650760  5469 raft_consensus.cc:748] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.650763  5470 raft_consensus.cc:748] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.651127  5469 consensus_queue.cc:261] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.651186  5470 consensus_queue.cc:261] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.653043  5468 tablet_bootstrap.cc:492] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:26.653216  5468 ts_tablet_manager.cc:1449] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.018s	user 0.003s	sys 0.000s
I20261118 02:55:26.653954  5468 raft_consensus.cc:360] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.654140  5468 raft_consensus.cc:386] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.654170  5469 ts_tablet_manager.cc:1480] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.000s
I20261118 02:55:26.654202  5468 raft_consensus.cc:748] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.654500  5468 consensus_queue.cc:261] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.654989  5468 ts_tablet_manager.cc:1480] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.002s	user 0.002s	sys 0.000s
I20261118 02:55:26.655587  5469 tablet_bootstrap.cc:492] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f: Bootstrap starting.
I20261118 02:55:26.658234  5468 tablet_bootstrap.cc:492] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8: Bootstrap starting.
I20261118 02:55:26.657541  5470 ts_tablet_manager.cc:1480] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.001s
I20261118 02:55:26.660919  5468 tablet_bootstrap.cc:654] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.663338  5470 tablet_bootstrap.cc:492] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:26.664065  5469 tablet_bootstrap.cc:654] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.673298  5470 tablet_bootstrap.cc:654] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.677798  5469 tablet_bootstrap.cc:492] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f: No bootstrap required, opened a new log
I20261118 02:55:26.677976  5469 ts_tablet_manager.cc:1449] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.005s
I20261118 02:55:26.678954  5469 raft_consensus.cc:360] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.679150  5469 raft_consensus.cc:386] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.679239  5469 raft_consensus.cc:748] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.679417  5470 tablet_bootstrap.cc:492] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:26.679479  5469 consensus_queue.cc:261] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.679586  5470 ts_tablet_manager.cc:1449] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.017s	user 0.002s	sys 0.003s
I20261118 02:55:26.680831  5468 tablet_bootstrap.cc:492] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8: No bootstrap required, opened a new log
I20261118 02:55:26.680965  5468 ts_tablet_manager.cc:1449] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent bootstrapping tablet: real 0.023s	user 0.002s	sys 0.010s
I20261118 02:55:26.681173  5469 ts_tablet_manager.cc:1480] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f: Time spent starting tablet: real 0.003s	user 0.002s	sys 0.001s
I20261118 02:55:26.681653  5468 raft_consensus.cc:360] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.681818  5468 raft_consensus.cc:386] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.681892  5468 raft_consensus.cc:748] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.682098  5468 consensus_queue.cc:261] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.683750  5468 ts_tablet_manager.cc:1480] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8: Time spent starting tablet: real 0.003s	user 0.001s	sys 0.001s
I20261118 02:55:26.685869  5470 raft_consensus.cc:360] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.686022  5470 raft_consensus.cc:386] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.686065  5470 raft_consensus.cc:748] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.686266  5470 consensus_queue.cc:261] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.689347  5470 ts_tablet_manager.cc:1480] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.001s
I20261118 02:55:26.689688  5470 tablet_bootstrap.cc:492] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef: Bootstrap starting.
I20261118 02:55:26.691386  5424 raft_consensus.cc:499] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:26.691532  5424 raft_consensus.cc:521] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.692039  5470 tablet_bootstrap.cc:654] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef: Neither blocks nor log segments found. Creating new log.
I20261118 02:55:26.692778  5424 leader_election.cc:290] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.693562  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b400925bbe6f4639b3d9e76cbb1ca681" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:26.693739  3751 raft_consensus.cc:2490] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:26.694175  3836 leader_election.cc:304] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:26.694469  5424 raft_consensus.cc:2826] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:26.694566  5424 raft_consensus.cc:499] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:26.694530  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b400925bbe6f4639b3d9e76cbb1ca681" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:26.694648  5424 raft_consensus.cc:3096] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.694712  3616 raft_consensus.cc:2490] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 0.
I20261118 02:55:26.695223  5470 tablet_bootstrap.cc:492] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef: No bootstrap required, opened a new log
I20261118 02:55:26.695348  5470 ts_tablet_manager.cc:1449] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent bootstrapping tablet: real 0.006s	user 0.003s	sys 0.001s
I20261118 02:55:26.696225  5470 raft_consensus.cc:360] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.696408  5470 raft_consensus.cc:386] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20261118 02:55:26.696491  5470 raft_consensus.cc:748] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Initialized, Role: FOLLOWER
I20261118 02:55:26.697541  5424 raft_consensus.cc:521] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.698520  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b400925bbe6f4639b3d9e76cbb1ca681" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:26.701494  3616 raft_consensus.cc:3096] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.698891  5424 leader_election.cc:290] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.696735  5470 consensus_queue.cc:261] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.698750  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b400925bbe6f4639b3d9e76cbb1ca681" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:26.702953  3751 raft_consensus.cc:3096] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.703986  5470 ts_tablet_manager.cc:1480] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.000s
I20261118 02:55:26.704284  3616 raft_consensus.cc:2490] T b400925bbe6f4639b3d9e76cbb1ca681 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:26.704876  3838 leader_election.cc:304] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:26.705199  5424 raft_consensus.cc:2826] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:26.705335  5424 raft_consensus.cc:705] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [term 1 LEADER]: Becoming Leader. State: Replica: 3f1973986ec1466eab5bec01dd81654f, State: Running, Role: LEADER
I20261118 02:55:26.705647  5424 consensus_queue.cc:238] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.706207  3751 raft_consensus.cc:2490] T b400925bbe6f4639b3d9e76cbb1ca681 P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f1973986ec1466eab5bec01dd81654f in term 1.
I20261118 02:55:26.710197  5424 consensus_queue.cc:1049] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:26.710708  5465 consensus_queue.cc:1049] T b400925bbe6f4639b3d9e76cbb1ca681 P 3f1973986ec1466eab5bec01dd81654f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20261118 02:55:26.772207  3690 connection.cc:573] server connection from 127.0.0.1:50108 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:26.772219  3837 connection.cc:573] client connection to 127.2.229.126:38025 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:26.772219  3838 connection.cc:573] server connection from 127.0.0.1:35078 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:26.772461  3690 connection.cc:573] client connection to 127.2.229.126:38025 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:26.772687  3956 connection.cc:573] client connection to 127.2.229.126:38025 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:26.772970  3553 connection.cc:573] client connection to 127.2.229.126:38025 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:26.773291  3553 connection.cc:573] server connection from 127.0.0.1:46360 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:26.773315  3956 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.229.126:38025)
W20261118 02:55:26.773818  3956 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.2.229.126:38025)
W20261118 02:55:26.774013  3956 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.2.229.126:38025)
W20261118 02:55:26.774312  3956 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.229.126:38025)
W20261118 02:55:26.774842  5464 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:26.773156 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.2.229.126:38025 (local address 127.2.229.67:54969)
1118 02:55:26.774066 (+   910us) negotiation.cc:107] Waiting for socket to connect
1118 02:55:26.774087 (+    21us) client_negotiation.cc:175] Beginning negotiation
1118 02:55:26.774217 (+   130us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 02:55:26.774803 (+   586us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":783}
W20261118 02:55:26.775566  5431 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:26.774633 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.2.229.126:38025 (local address 127.2.229.66:38331)
1118 02:55:26.774905 (+   272us) negotiation.cc:107] Waiting for socket to connect
1118 02:55:26.774922 (+    17us) client_negotiation.cc:175] Beginning negotiation
1118 02:55:26.775038 (+   116us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 02:55:26.775526 (+   488us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":175}
I20261118 02:55:26.789143  5223 raft_consensus.cc:499] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:26.789335  5223 raft_consensus.cc:521] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.789904  5223 leader_election.cc:290] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.790370  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c46a0eff7633462fb3719143c1f7cfbb" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:26.790398  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c46a0eff7633462fb3719143c1f7cfbb" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:26.790587  3901 raft_consensus.cc:2490] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:26.790592  3751 raft_consensus.cc:2490] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:26.790972  3553 leader_election.cc:304] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:26.791229  5223 raft_consensus.cc:2826] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:26.791308  5223 raft_consensus.cc:499] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:26.791404  5223 raft_consensus.cc:3096] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.794036  5223 raft_consensus.cc:521] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.794605  5223 leader_election.cc:290] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.794925  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c46a0eff7633462fb3719143c1f7cfbb" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:26.795081  3901 raft_consensus.cc:3096] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.795058  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c46a0eff7633462fb3719143c1f7cfbb" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:26.795241  3751 raft_consensus.cc:3096] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.797178  3901 raft_consensus.cc:2490] T c46a0eff7633462fb3719143c1f7cfbb P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:26.797281  3751 raft_consensus.cc:2490] T c46a0eff7633462fb3719143c1f7cfbb P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:26.797603  3553 leader_election.cc:304] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:26.797863  5223 raft_consensus.cc:2826] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:26.798029  5223 raft_consensus.cc:705] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:26.798347  5223 consensus_queue.cc:238] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.805913  5223 consensus_queue.cc:1049] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:26.806227  5491 consensus_queue.cc:1049] T c46a0eff7633462fb3719143c1f7cfbb P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:26.819752  5471 raft_consensus.cc:499] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:26.819947  5471 raft_consensus.cc:521] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.820597  5471 leader_election.cc:290] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.821031  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d8766c4d56fa4689a69b153ea7a613ee" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:26.821228  3751 raft_consensus.cc:2490] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:26.821592  3552 leader_election.cc:304] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:26.821857  5471 raft_consensus.cc:2826] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:26.821936  5471 raft_consensus.cc:499] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:26.822029  5471 raft_consensus.cc:3096] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.823872  5471 raft_consensus.cc:521] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.824396  5471 leader_election.cc:290] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.824501  3901 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d8766c4d56fa4689a69b153ea7a613ee" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:26.824764  3901 raft_consensus.cc:2490] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 0.
I20261118 02:55:26.825029  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d8766c4d56fa4689a69b153ea7a613ee" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:26.825215  3902 raft_consensus.cc:3096] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.825284  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "d8766c4d56fa4689a69b153ea7a613ee" candidate_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef"
I20261118 02:55:26.825453  3751 raft_consensus.cc:3096] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:26.827188  3902 raft_consensus.cc:2490] T d8766c4d56fa4689a69b153ea7a613ee P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:26.827325  3751 raft_consensus.cc:2490] T d8766c4d56fa4689a69b153ea7a613ee P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fdf7d7ece324fefb3a0c376add5b7e8 in term 1.
I20261118 02:55:26.827428  5476 raft_consensus.cc:499] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:26.827628  3553 leader_election.cc:304] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 3f1973986ec1466eab5bec01dd81654f; no voters: 
I20261118 02:55:26.827564  5476 raft_consensus.cc:521] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:26.827956  5471 raft_consensus.cc:2826] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:26.828289  5476 leader_election.cc:290] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:26.828122  5471 raft_consensus.cc:705] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1fdf7d7ece324fefb3a0c376add5b7e8, State: Running, Role: LEADER
I20261118 02:55:26.828703  5471 consensus_queue.cc:238] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:26.828722  3751 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b119ead4bb6548b4a37da02ed6daa6a8" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" is_pre_election: true
I20261118 02:55:26.831491  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "b119ead4bb6548b4a37da02ed6daa6a8" candidate_uuid: "3f1973986ec1466eab5bec01dd81654f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:26.832070  3838 leader_election.cc:304] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f1973986ec1466eab5bec01dd81654f; no voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:26.832361  5476 raft_consensus.cc:2771] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20261118 02:55:26.832858  5223 consensus_queue.cc:1049] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:26.833478  5223 consensus_queue.cc:1049] T d8766c4d56fa4689a69b153ea7a613ee P 1fdf7d7ece324fefb3a0c376add5b7e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:26.936182  5518 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:55:26.936419  5518 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:26.937175  5518 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:26.939237  5342 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:26.939416  5342 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:26.939498  5342 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:26.939752  5518 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 02:55:27.056810  5508 raft_consensus.cc:499] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20261118 02:55:27.057022  5508 raft_consensus.cc:521] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:27.057701  5508 leader_election.cc:290] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:27.058296  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c1f0147a1efb4f4f813478f119043bec" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" is_pre_election: true
I20261118 02:55:27.058320  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c1f0147a1efb4f4f813478f119043bec" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f" is_pre_election: true
I20261118 02:55:27.058497  3616 raft_consensus.cc:2490] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:27.058538  3902 raft_consensus.cc:2490] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 0.
I20261118 02:55:27.058902  3691 leader_election.cc:304] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:27.059204  5508 raft_consensus.cc:2826] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20261118 02:55:27.059333  5508 raft_consensus.cc:499] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20261118 02:55:27.059420  5508 raft_consensus.cc:3096] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:27.062515  5508 raft_consensus.cc:521] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:27.063611  5508 leader_election.cc:290] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Requested vote from peers 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:27.063746  3616 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c1f0147a1efb4f4f813478f119043bec" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8"
I20261118 02:55:27.063983  3616 raft_consensus.cc:3096] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:27.064313  3902 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "c1f0147a1efb4f4f813478f119043bec" candidate_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1973986ec1466eab5bec01dd81654f"
I20261118 02:55:27.064481  3902 raft_consensus.cc:3096] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
I20261118 02:55:27.066200  3616 raft_consensus.cc:2490] T c1f0147a1efb4f4f813478f119043bec P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:27.066627  3691 leader_election.cc:304] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8, 99a39b1d9d814deeb3a8a644e9b392ef; no voters: 
I20261118 02:55:27.066928  5508 raft_consensus.cc:2826] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 FOLLOWER]: Leader election won for term 1
I20261118 02:55:27.067077  5508 raft_consensus.cc:705] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [term 1 LEADER]: Becoming Leader. State: Replica: 99a39b1d9d814deeb3a8a644e9b392ef, State: Running, Role: LEADER
I20261118 02:55:27.067217  3902 raft_consensus.cc:2490] T c1f0147a1efb4f4f813478f119043bec P 3f1973986ec1466eab5bec01dd81654f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99a39b1d9d814deeb3a8a644e9b392ef in term 1.
I20261118 02:55:27.067411  5508 consensus_queue.cc:238] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } }
I20261118 02:55:27.070578  5508 consensus_queue.cc:1049] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:27.070902  5507 consensus_queue.cc:1049] T c1f0147a1efb4f4f813478f119043bec P 99a39b1d9d814deeb3a8a644e9b392ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20261118 02:55:27.478394  2965 external_mini_cluster.cc:1372] Running /tmp/dist-test-task8fD3U4/build/asan/bin/kudu
/tmp/dist-test-task8fD3U4/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.124:36695
--webserver_interface=127.2.229.124
--webserver_port=39557
--builtin_ntp_servers=127.2.229.84:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20261118 02:55:27.614560  5531 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:55:27.614765  5531 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:27.615352  5531 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
W20261118 02:55:27.617887  5342 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:27.618136  5342 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695): Network error: Client connection negotiation failed: client connection to 127.2.229.124:36695: connect: Connection refused (error 111)
I20261118 02:55:27.618258  5342 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 813a6937cea442ef81645fccc32207f4
I20261118 02:55:27.618496  5531 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20261118 02:55:27.684228  3902 raft_consensus.cc:3096] T b119ead4bb6548b4a37da02ed6daa6a8 P 3f1973986ec1466eab5bec01dd81654f [term 0 FOLLOWER]: Advancing to term 1
W20261118 02:55:27.833762  5526 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20261118 02:55:27.834057  5526 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20261118 02:55:27.834132  5526 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20261118 02:55:27.843071  5526 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20261118 02:55:27.843153  5526 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20261118 02:55:27.843206  5526 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20261118 02:55:27.843257  5526 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20261118 02:55:27.853453  5526 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.229.84:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:39081
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.229.124:36695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.229.124
--webserver_port=39557
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 3458f03c6ee765ff81fe1ff179b7d6d82b82420b
build type FASTDEBUG
built by None at 18 Nov 2026 02:43:21 UTC on bdcb31816ec0
build id 14837
ASAN enabled
I20261118 02:55:27.855036  5526 env_posix.cc:2314] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20261118 02:55:27.856874  5526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20261118 02:55:27.865374  5544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:27.865410  5545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20261118 02:55:27.865906  5547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20261118 02:55:27.866293  5526 server_base.cc:1061] running on GCE node
I20261118 02:55:27.867000  5526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20261118 02:55:27.868415  5526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20261118 02:55:27.869627  5526 hybrid_clock.cc:648] HybridClock initialized: now 1794970527869583 us; error 26 us; skew 500 ppm
I20261118 02:55:27.869941  5526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20261118 02:55:27.872119  5526 webserver.cc:577] Webserver started at http://127.2.229.124:39557/ using document root <none> and password file <none>
I20261118 02:55:27.872843  5526 fs_manager.cc:362] Metadata directory not provided
I20261118 02:55:27.872949  5526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20261118 02:55:27.878437  5526 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.000s	sys 0.005s
I20261118 02:55:27.881456  5553 log_block_manager.cc:4130] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20261118 02:55:27.882910  5526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20261118 02:55:27.883075  5526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data,/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
uuid: "813a6937cea442ef81645fccc32207f4"
format_stamp: "Formatted at 2026-11-18 02:54:57 on dist-test-slave-49p1"
I20261118 02:55:27.883770  5526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20261118 02:55:27.903831  5526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20261118 02:55:27.904734  5526 env_posix.cc:2314] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20261118 02:55:27.905011  5526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20261118 02:55:27.923594  5526 rpc_server.cc:307] RPC server started. Bound to: 127.2.229.124:36695
I20261118 02:55:27.923611  5604 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.2.229.124:36695 every 8 connection(s)
I20261118 02:55:27.925367  5526 server_base.cc:1193] Dumped server information to /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/master-2/data/info.pb
I20261118 02:55:27.926820  2965 external_mini_cluster.cc:1434] Started /tmp/dist-test-task8fD3U4/build/asan/bin/kudu as pid 5526
I20261118 02:55:27.934008  5605 sys_catalog.cc:263] Verifying existing consensus state
I20261118 02:55:27.937644  5605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap starting.
I20261118 02:55:27.959455  5605 log.cc:826] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Log is configured to *not* fsync() on all Append() calls
I20261118 02:55:28.042068  5605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap replayed 1/1 log segments. Stats: ops{read=113 overwritten=0 applied=113 ignored=0} inserts{seen=75 ignored=0} mutations{seen=100 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20261118 02:55:28.042614  5605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: Bootstrap complete.
I20261118 02:55:28.049194  5605 raft_consensus.cc:360] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:28.049918  5605 raft_consensus.cc:748] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 813a6937cea442ef81645fccc32207f4, State: Initialized, Role: FOLLOWER
I20261118 02:55:28.050611  5605 consensus_queue.cc:261] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 113, Last appended: 8.113, Last appended by leader: 113, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:28.051461  5610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:28.051775  5610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:28.052516  5605 sys_catalog.cc:565] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: configured and running, proceeding with master startup.
I20261118 02:55:28.072705  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:28.078828  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:28.081604  5624 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:28.081689  5624 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: loading cluster ID for follower catalog manager: success
I20261118 02:55:28.085057  5624 catalog_manager.cc:1625] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: acquiring CA information for follower catalog manager: success
I20261118 02:55:28.087491  5624 catalog_manager.cc:1653] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20261118 02:55:28.097249  3664 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:55:28.097417  3664 heartbeater.cc:500] Master 127.2.229.124:36695 was elected leader, sending a full tablet report...
I20261118 02:55:28.101508  5569 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" instance_seqno: 1794970498532688) as {username='slave'} at 127.2.229.65:45763; Asking this server to re-register.
I20261118 02:55:28.102363  3664 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:28.104434  5569 ts_manager.cc:210] Registered new tserver with Master: 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705)
I20261118 02:55:28.136616  5491 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:55:28.136852  5491 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:28.137568  5491 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:28.138643  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:55:28.138813  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:28.138906  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:28.139163  5491 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:28.197311  5630 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:55:28.197511  5630 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:28.198184  5630 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:28.203583  3801 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
I20261118 02:55:28.210520  5570 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" instance_seqno: 1794970499028671) as {username='slave'} at 127.2.229.66:36123; Asking this server to re-register.
I20261118 02:55:28.211706  3801 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:28.213747  5570 ts_manager.cc:210] Registered new tserver with Master: 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377)
I20261118 02:55:28.219092  3948 heartbeater.cc:345] Connected to a master server at 127.2.229.124:36695
W20261118 02:55:28.219417  5342 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:28.221925  5569 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" instance_seqno: 1794970499527172) as {username='slave'} at 127.2.229.67:59991; Asking this server to re-register.
I20261118 02:55:28.222345  5580 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 10 candidate_status { last_received { term: 9 index: 117 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4" is_pre_election: true
I20261118 02:55:28.222728  3948 heartbeater.cc:462] Registering TS with master...
I20261118 02:55:28.222995  5580 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 457eb38cba564ecd9f6f11245172260b in term 8.
I20261118 02:55:28.223645  5342 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d
I20261118 02:55:28.223961  5630 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Leader pre-election won for term 10
I20261118 02:55:28.224077  5630 raft_consensus.cc:499] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Starting leader election (detected failure of leader 26c6b20fa203404a88463ea27bdd3c7d)
I20261118 02:55:28.224187  5630 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 9 FOLLOWER]: Advancing to term 10
I20261118 02:55:28.224674  5569 ts_manager.cc:210] Registered new tserver with Master: 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:28.226445  5630 raft_consensus.cc:521] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:28.227042  5630 leader_election.cc:290] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 election: Requested vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 813a6937cea442ef81645fccc32207f4 (127.2.229.124:36695)
I20261118 02:55:28.227790  5580 tablet_service.cc:1906] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "457eb38cba564ecd9f6f11245172260b" candidate_term: 10 candidate_status { last_received { term: 9 index: 117 } } ignore_live_leader: false dest_uuid: "813a6937cea442ef81645fccc32207f4"
I20261118 02:55:28.227967  5580 raft_consensus.cc:3096] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 8 FOLLOWER]: Advancing to term 10
I20261118 02:55:28.232302  5580 raft_consensus.cc:2490] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 457eb38cba564ecd9f6f11245172260b in term 10.
I20261118 02:55:28.232748  5342 leader_election.cc:304] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 457eb38cba564ecd9f6f11245172260b, 813a6937cea442ef81645fccc32207f4; no voters: 
I20261118 02:55:28.233035  5630 raft_consensus.cc:2826] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 10 FOLLOWER]: Leader election won for term 10
W20261118 02:55:28.233604  5342 leader_election.cc:336] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:28.236358  5630 raft_consensus.cc:705] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [term 10 LEADER]: Becoming Leader. State: Replica: 457eb38cba564ecd9f6f11245172260b, State: Running, Role: LEADER
I20261118 02:55:28.236807  5630 consensus_queue.cc:238] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 117, Committed index: 117, Last appended: 9.117, Last appended by leader: 117, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:28.238528  5636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 10 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:28.241077  5580 raft_consensus.cc:1285] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Refusing update from remote peer 457eb38cba564ecd9f6f11245172260b: Log matching property violated. Preceding OpId in replica: term: 8 index: 113. Preceding OpId from leader: term: 10 index: 118. (index mismatch)
I20261118 02:55:28.243892  5638 consensus_queue.cc:1049] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [LEADER]: Connected to new peer: Peer: permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 118, Last known committed idx: 113, Time since last communication: 0.000s
I20261118 02:55:28.249001  5636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
W20261118 02:55:28.249681  5342 consensus_peers.cc:597] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b -> Peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Couldn't send request to peer 26c6b20fa203404a88463ea27bdd3c7d. Status: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:28.254060  5642 catalog_manager.cc:1526] Loading table and tablet metadata into memory...
I20261118 02:55:28.257992  5642 catalog_manager.cc:694] Loaded metadata for table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190]
I20261118 02:55:28.258582  5642 catalog_manager.cc:694] Loaded metadata for table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c]
I20261118 02:55:28.259119  5641 mvcc.cc:204] Tried to move back new op lower bound from 7352199276076613632 to 7352199275831820288. Current Snapshot: MvccSnapshot[applied={T|T < 7352199276076613632 or (T in {7352199276076613632})}]
I20261118 02:55:28.259443  5642 catalog_manager.cc:694] Loaded metadata for table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6]
I20261118 02:55:28.259922  5642 catalog_manager.cc:694] Loaded metadata for table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73]
I20261118 02:55:28.260382  5642 catalog_manager.cc:694] Loaded metadata for table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e]
I20261118 02:55:28.260816  5642 catalog_manager.cc:694] Loaded metadata for table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b]
I20261118 02:55:28.260901  5610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 457eb38cba564ecd9f6f11245172260b. Latest consensus state: current_term: 10 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:28.261139  5610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:28.261464  5642 catalog_manager.cc:694] Loaded metadata for table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da]
I20261118 02:55:28.261788  5642 catalog_manager.cc:694] Loaded metadata for table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66]
I20261118 02:55:28.262087  5642 catalog_manager.cc:694] Loaded metadata for table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d]
I20261118 02:55:28.262394  5642 catalog_manager.cc:694] Loaded metadata for table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db]
I20261118 02:55:28.262754  5642 catalog_manager.cc:694] Loaded metadata for table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf]
I20261118 02:55:28.262694  5644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:28.262990  5644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:28.263064  5642 catalog_manager.cc:694] Loaded metadata for table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b]
I20261118 02:55:28.263397  5642 catalog_manager.cc:694] Loaded metadata for table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d]
I20261118 02:55:28.263708  5642 catalog_manager.cc:694] Loaded metadata for table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7]
I20261118 02:55:28.264014  5642 catalog_manager.cc:694] Loaded metadata for table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67]
I20261118 02:55:28.264436  5642 catalog_manager.cc:694] Loaded metadata for table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d]
I20261118 02:55:28.264829  5642 catalog_manager.cc:694] Loaded metadata for table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec]
I20261118 02:55:28.265306  5642 catalog_manager.cc:694] Loaded metadata for table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1]
I20261118 02:55:28.265707  5642 catalog_manager.cc:694] Loaded metadata for table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f]
I20261118 02:55:28.266211  5642 catalog_manager.cc:694] Loaded metadata for table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139]
I20261118 02:55:28.266783  5644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 10 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:28.266963  5644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [sys.catalog]: This master's current role is: FOLLOWER
I20261118 02:55:28.269320  5642 tablet_loader.cc:96] loaded metadata for tablet 0e901a03b90e4c978eb98be87163cfe5 (table default.table_d4fec0aaed15433ea373740872db53d9 [id=00713c8ca8db4bf9a860d32614f99190])
I20261118 02:55:28.269738  5642 tablet_loader.cc:96] loaded metadata for tablet 1904140da78b4567a9decb250c6f74a3 (table default.table_61b6eaac749b4be0b0ddfba6ff9aa706 [id=e0ed3867fcbb42a4b0133f38e0372a6f])
I20261118 02:55:28.270381  5642 tablet_loader.cc:96] loaded metadata for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc (table default.table_a4f08da750c54c49b9b7d73ebcf52624 [id=790b0571767b49eead05512b819d27db])
I20261118 02:55:28.270735  5642 tablet_loader.cc:96] loaded metadata for tablet 2c92955aa65142f1bd20f4b8f3321500 (table default.table_38304cbb980440c2a8b8a6c13f49949c [id=7140b942170b4483b0e01e32d5ba5e6d])
I20261118 02:55:28.270915  5630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 10 leader_uuid: "457eb38cba564ecd9f6f11245172260b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } } }
I20261118 02:55:28.271173  5630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b [sys.catalog]: This master's current role is: LEADER
I20261118 02:55:28.271363  5642 tablet_loader.cc:96] loaded metadata for tablet 462a9be0ae114b01a9c69416185803f8 (table default.table_571d1360811c42dc815efb439b2b52e6 [id=e797fad7075b40d9bdd674a0e0023139])
I20261118 02:55:28.271699  5642 tablet_loader.cc:96] loaded metadata for tablet 46f05790dbdc4b8185a7b0362f604cf8 (table default.table_28c62f411da74545aa42dad7325bc064 [id=7d99cc94d0ab4bf9a21881fc6dcf91bf])
I20261118 02:55:28.273630  5642 tablet_loader.cc:96] loaded metadata for tablet 72f427c72a554ac1a0a2f2c3a78865da (table default.table_30f082fbe850451b80d98df75ed1081a [id=44b390866720455b8d2714eae004383b])
I20261118 02:55:28.273978  5642 tablet_loader.cc:96] loaded metadata for tablet 78189cccd457466bb5d01231038fec24 (table default.table_a7f3710d39fb447da94e16d4ab3933fd [id=4a09f210503b499f8e842f6daafb33da])
I20261118 02:55:28.274513  5642 tablet_loader.cc:96] loaded metadata for tablet 8bed657d0da64d95851f8a86216b6c2c (table default.table_6e09d0e89581458183ed524cc1404849 [id=01b05aad48954b8fb2aa6d7b31a6578c])
I20261118 02:55:28.275988  5642 tablet_loader.cc:96] loaded metadata for tablet a67bc60c7ecb4b7e8e03e04267bb91a5 (table default.table_0385cef6593c419eb426657c8cbef2bf [id=8a5ea4b95bee4db7af9ba875250d564b])
I20261118 02:55:28.276566  5642 tablet_loader.cc:96] loaded metadata for tablet ab61b43e639a4e75bc2e7009ea86a64f (table default.table_f0ae490a0df64e1097e144c7a370df61 [id=a59b9425c01049269883cf7767936c67])
I20261118 02:55:28.276911  5642 tablet_loader.cc:96] loaded metadata for tablet b119ead4bb6548b4a37da02ed6daa6a8 (table default.table_287099a68a3540f1be0284c71d4d7056 [id=4e5b6eaa70d34daf8a68fc17ec1afb66])
I20261118 02:55:28.277233  5642 tablet_loader.cc:96] loaded metadata for tablet b129518ce23c45758ebcaa55410757d8 (table default.table_30b09a2798a54b01be7618b8a28eafaf [id=dd0f01caa36a4d3e9795daaf698b1ce1])
I20261118 02:55:28.277590  5642 tablet_loader.cc:96] loaded metadata for tablet b400925bbe6f4639b3d9e76cbb1ca681 (table default.table_4f7796d5d5d54def843b971e66d15ea6 [id=a2ece7af36934cdf80ad8458024adec7])
I20261118 02:55:28.278465  5642 tablet_loader.cc:96] loaded metadata for tablet c1f0147a1efb4f4f813478f119043bec (table default.table_a14cb6414555446286bf35bc5c6fb910 [id=0ce698410f004edaa9ad4ba1afe2de73])
I20261118 02:55:28.278752  5642 tablet_loader.cc:96] loaded metadata for tablet c46a0eff7633462fb3719143c1f7cfbb (table default.table_22f31dd1ff9e4185a6618f11b5493434 [id=aea45564ce10469393d3c7cc03b0bb9d])
I20261118 02:55:28.279328  5642 tablet_loader.cc:96] loaded metadata for tablet ccf78258c9634540a178855a142f4d4b (table default.table_2e3917ff98f2474984580e61caa9c17b [id=2584489d44ac4c6b90ca7c9f390c617e])
I20261118 02:55:28.279902  5642 tablet_loader.cc:96] loaded metadata for tablet d8766c4d56fa4689a69b153ea7a613ee (table default.table_87d592a09c3348028719a603956fbce6 [id=9724ed64fa5043bcbfee1caefac8e86d])
I20261118 02:55:28.280303  5642 tablet_loader.cc:96] loaded metadata for tablet e71c5cd71ebf4fa6bd158d59ed2b191d (table default.table_8ae376d4d4d64414b5c08dc3c2e58bc4 [id=052f3134115244a6a2443e96121313e6])
I20261118 02:55:28.281126  5642 tablet_loader.cc:96] loaded metadata for tablet fbdc4c7b574e4f94b816fb0c0b14c185 (table default.table_c113c2b57e17479e92e241234a241af7 [id=da0cb25ccfed426ca61f0dd9f93d30ec])
I20261118 02:55:28.281781  5642 catalog_manager.cc:1535] Initializing Kudu cluster ID...
I20261118 02:55:28.283506  5642 catalog_manager.cc:1310] Loaded cluster ID: 4547b4c2697449c8bc7f957ed454ac1e
I20261118 02:55:28.283587  5642 catalog_manager.cc:1546] Initializing Kudu internal certificate authority...
I20261118 02:55:28.285229  5642 catalog_manager.cc:1555] Loading token signing keys...
I20261118 02:55:28.286448  5642 catalog_manager.cc:6232] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Loaded TSK: 0
I20261118 02:55:28.287295  5642 catalog_manager.cc:1565] Initializing in-progress tserver states...
I20261118 02:55:28.288455  5491 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:55:28.288620  5642 catalog_manager.cc:1578] Loading latest processed Hive Metastore notification log event ID...
I20261118 02:55:28.288666  5491 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:28.289443  5491 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
I20261118 02:55:28.290257  5642 catalog_manager.cc:6070] Last processed Hive Metastore notification event ID: 20
W20261118 02:55:28.290436  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:55:28.290585  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:28.290745  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:28.291020  5491 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:28.344012  5356 master_service.cc:965] ReplaceTablet: received request to replace tablet 0e901a03b90e4c978eb98be87163cfe5 from {username='slave'} at 127.0.0.1:49658
I20261118 02:55:28.344053  5354 master_service.cc:965] ReplaceTablet: received request to replace tablet 2b01d7e596bd4790b8ba5dc88d8e12cc from {username='slave'} at 127.0.0.1:49658
I20261118 02:55:28.347713  5353 master_service.cc:965] ReplaceTablet: received request to replace tablet d8766c4d56fa4689a69b153ea7a613ee from {username='slave'} at 127.0.0.1:49658
I20261118 02:55:28.347838  5350 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49658:
table { table_name: "default.Table_2e3917ff98f2474984580e61caa9c17b" } new_table_name: "default.Table_3c225da1f7fa435083909e053ec9751a" modify_external_catalogs: true
I20261118 02:55:28.347947  5348 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49658:
table { table_name: "default.Table_8ae376d4d4d64414b5c08dc3c2e58bc4" } new_table_name: "default.Table_5091d82298ca44bea5335d4989eb02d8" modify_external_catalogs: true
I20261118 02:55:28.347772  5351 catalog_manager.cc:2548] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49658:
table { table_name: "default.Table_87d592a09c3348028719a603956fbce6" } modify_external_catalogs: true
I20261118 02:55:28.348358  5349 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49658:
table { table_name: "default.Table_a14cb6414555446286bf35bc5c6fb910" } new_table_name: "default.Table_324a2600fef947748ef03e27f7cb7e09" modify_external_catalogs: true
I20261118 02:55:28.348433  5351 catalog_manager.cc:2796] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49658:
table { table_name: "default.Table_87d592a09c3348028719a603956fbce6" } modify_external_catalogs: true
I20261118 02:55:28.348896  5352 master_service.cc:965] ReplaceTablet: received request to replace tablet 2c92955aa65142f1bd20f4b8f3321500 from {username='slave'} at 127.0.0.1:49658
I20261118 02:55:28.350191  5347 catalog_manager.cc:2298] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49658:
name: "default.Table_dc982811b980416484416db1f74c685d"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20261118 02:55:28.355518  5355 master_service.cc:965] ReplaceTablet: received request to replace tablet ccf78258c9634540a178855a142f4d4b from {username='slave'} at 127.0.0.1:49658
F20261118 02:55:28.359395  5409 hms_notification_log_listener.cc:287] non-monotonous event IDs from HMS: current 21, previous 21; dumping first 2 out of 15 received events: NotificationEvent(eventId=21, eventTime=1794970503, eventType=CREATE_TABLE, dbName=default, tableName=table_f0ae490a0df64e1097e144c7a370df61, message={"server":"","servicePrincipal":"","db":"default","table":"table_f0ae490a0df64e1097e144c7a370df61","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_f0ae490a0df64e1097e144c7a370df61\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970503},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_f0ae490a0df64e1097e144c7a370df61\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_f0ae490a0df64e1097e144c7a370df61\",\"kudu.table_id\":\"a59b9425c01049269883cf7767936c67\",\"transient_lastDdlTime\":\"1794970503\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"14\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503,"files":[]}, messageFormat=json-0.2, catName=hive); NotificationEvent(eventId=21, eventTime=1794970503, eventType=DROP_TABLE, dbName=default, tableName=table_c113c2b57e17479e92e241234a241af7, message={"server":"","servicePrincipal":"","db":"default","table":"table_c113c2b57e17479e92e241234a241af7","tableType":"MANAGED_TABLE","tableObjJson":"{\"1\":{\"str\":\"table_c113c2b57e17479e92e241234a241af7\"},\"2\":{\"str\":\"default\"},\"3\":{\"str\":\"slave\"},\"4\":{\"i32\":1794970500},\"5\":{\"i32\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"lst\":[\"rec\",1,{\"1\":{\"str\":\"key\"},\"2\":{\"str\":\"int\"},\"3\":{\"str\":\"\"}}]},\"2\":{\"str\":\"file:/tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/warehouse/table_c113c2b57e17479e92e241234a241af7\"},\"3\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduInputFormat\"},\"4\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduOutputFormat\"},\"5\":{\"tf\":0},\"6\":{\"i32\":0},\"7\":{\"rec\":{\"1\":{\"str\":\"\"},\"2\":{\"str\":\"org.apache.hadoop.hive.kudu.KuduSerDe\"},\"3\":{\"map\":[\"str\",\"str\",0,{}]}}},\"8\":{\"lst\":[\"str\",0]},\"9\":{\"lst\":[\"rec\",0]},\"10\":{\"map\":[\"str\",\"str\",0,{}]},\"11\":{\"rec\":{\"1\":{\"lst\":[\"str\",0]},\"2\":{\"lst\":[\"lst\",0]},\"3\":{\"map\":[\"lst\",\"str\",0,{}]}}},\"12\":{\"tf\":0}}},\"8\":{\"lst\":[\"rec\",0]},\"9\":{\"map\":[\"str\",\"str\",7,{\"kudu.table_name\":\"default.table_c113c2b57e17479e92e241234a241af7\",\"kudu.table_id\":\"da0cb25ccfed426ca61f0dd9f93d30ec\",\"transient_lastDdlTime\":\"1794970500\",\"comment\":\"\",\"kudu.master_addresses\":\"127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695\",\"storage_handler\":\"org.apache.hadoop.hive.kudu.KuduStorageHandler\",\"kudu.cluster_id\":\"4547b4c2697449c8bc7f957ed454ac1e\"}]},\"10\":{\"str\":\"\"},\"11\":{\"str\":\"\"},\"12\":{\"str\":\"MANAGED_TABLE\"},\"15\":{\"tf\":0},\"17\":{\"str\":\"hive\"},\"18\":{\"i32\":1}}","timestamp":1794970503}, messageFormat=json-0.2, catName=hive);
*** Check failure stack trace: ***
I20261118 02:55:28.362301  5356 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet 0e901a03b90e4c978eb98be87163cfe5
    @     0x7f93d6efed5d  google::LogMessage::Fail() at ??:0
I20261118 02:55:28.364419  5356 catalog_manager.cc:6936] ReplaceTablet: tablet 0e901a03b90e4c978eb98be87163cfe5 deleted and replaced by tablet f17bfd60c79f46efa6b801c3cb892ec0
    @     0x7f93d6f02b23  google::LogMessage::SendToLog() at ??:0
    @     0x7f93d6efe75c  google::LogMessage::Flush() at ??:0
I20261118 02:55:28.366576  5356 catalog_manager.cc:3585] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49658:
table { table_name: "default.Table_0385cef6593c419eb426657c8cbef2bf" } new_table_name: "default.Table_496441d99faa4e2b8c7cd98e4a5ce745" modify_external_catalogs: true
    @     0x7f93d6efe9a1  google::LogMessage::~LogMessage() at ??:0
I20261118 02:55:28.374794  5354 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet 2b01d7e596bd4790b8ba5dc88d8e12cc
I20261118 02:55:28.375406  5354 catalog_manager.cc:6936] ReplaceTablet: tablet 2b01d7e596bd4790b8ba5dc88d8e12cc deleted and replaced by tablet 96c806f34d0643b68afdae4e9eaf3498
    @     0x7f93ec37e9a4  kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
I20261118 02:55:28.388564  3597 tablet_service.cc:1547] Processing DeleteTablet for tablet 0e901a03b90e4c978eb98be87163cfe5 with delete_type TABLET_DATA_DELETED (replaced by tablet f17bfd60c79f46efa6b801c3cb892ec0) from {username='slave'} at 127.0.0.1:46376
I20261118 02:55:28.388963  3595 tablet_service.cc:1547] Processing DeleteTablet for tablet 2b01d7e596bd4790b8ba5dc88d8e12cc with delete_type TABLET_DATA_DELETED (replaced by tablet 96c806f34d0643b68afdae4e9eaf3498) from {username='slave'} at 127.0.0.1:46376
    @     0x7f93ec37d251  kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
    @     0x7f93ec384ecc  kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
    @     0x7f93ec384d49  std::_Function_handler<>::_M_invoke() at ??:0
    @           0x372c7d  std::function<>::operator()() at /home/jenkins-slave/workspace/build_and_test/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_stacktrace.h:115
    @     0x7f93d8347dc9  kudu::Thread::SuperviseThread() at ??:0
I20261118 02:55:28.415970  5353 catalog_manager.cc:6135] T 00000000000000000000000000000000 P 457eb38cba564ecd9f6f11245172260b: Sending DeleteTablet for 3 replicas of tablet d8766c4d56fa4689a69b153ea7a613ee
I20261118 02:55:28.416579  5353 catalog_manager.cc:6936] ReplaceTablet: tablet d8766c4d56fa4689a69b153ea7a613ee deleted and replaced by tablet f5511619f39f4deba7ca2cf13ad37258
    @     0x7f93dba226db  start_thread at ??:0
    @     0x7f93d449261f  clone at ??:0
I20261118 02:55:28.420300  5657 tablet_replica.cc:333] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:28.420519  5657 raft_consensus.cc:2261] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:28.420724  5657 raft_consensus.cc:2290] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:28.421166  5657 ts_tablet_manager.cc:1962] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:28.426870  5657 ts_tablet_manager.cc:1975] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:28.426992  5657 log.cc:1199] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/0e901a03b90e4c978eb98be87163cfe5
I20261118 02:55:28.427407  5657 ts_tablet_manager.cc:1996] T 0e901a03b90e4c978eb98be87163cfe5 P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
I20261118 02:55:28.431715  5657 tablet_replica.cc:333] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: stopping tablet replica
I20261118 02:55:28.432021  5657 raft_consensus.cc:2261] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20261118 02:55:28.432227  5657 raft_consensus.cc:2290] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20261118 02:55:28.432813  5657 ts_tablet_manager.cc:1962] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20261118 02:55:28.439225  5657 ts_tablet_manager.cc:1975] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20261118 02:55:28.439370  5657 log.cc:1199] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting WAL directory at /tmp/dist-test-task8fD3U4/test-tmp/master-stress-test.2.CatalogManagerConfigurations_MasterStressTest.Test_2.1794970481606018-2965-0/minicluster-data/ts-0/wal/wals/2b01d7e596bd4790b8ba5dc88d8e12cc
I20261118 02:55:28.439713  5657 ts_tablet_manager.cc:1996] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 1fdf7d7ece324fefb3a0c376add5b7e8: Deleting consensus metadata
W20261118 02:55:28.624691  3551 connection.cc:573] server connection from 127.0.0.1:46376 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:28.624712  3552 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:28.624949  5650 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:28.365265 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:35090 (local address 127.2.229.67:42237)
1118 02:55:28.412363 (+ 47098us) server_negotiation.cc:207] Beginning negotiation
1118 02:55:28.412369 (+     6us) server_negotiation.cc:400] Waiting for connection header
1118 02:55:28.412391 (+    22us) server_negotiation.cc:408] Connection header received
1118 02:55:28.412463 (+    72us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
1118 02:55:28.412468 (+     5us) server_negotiation.cc:462] Received NEGOTIATE request from client
1118 02:55:28.412546 (+    78us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
1118 02:55:28.419973 (+  7427us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
1118 02:55:28.624911 (+204938us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35090: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"server-negotiator.queue_time_us":46974,"thread_start_us":55089,"threads_started":1}
W20261118 02:55:28.625101  3689 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:28.625598  5654 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:28.376008 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.2.229.66:40377)
1118 02:55:28.412964 (+ 36956us) server_negotiation.cc:207] Beginning negotiation
1118 02:55:28.412970 (+     6us) server_negotiation.cc:400] Waiting for connection header
1118 02:55:28.412989 (+    19us) server_negotiation.cc:408] Connection header received
1118 02:55:28.413060 (+    71us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
1118 02:55:28.413066 (+     6us) server_negotiation.cc:462] Received NEGOTIATE request from client
1118 02:55:28.413136 (+    70us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
1118 02:55:28.413253 (+   117us) server_negotiation.cc:227] Negotiated authn=CERTIFICATE
1118 02:55:28.420292 (+  7039us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
1118 02:55:28.421747 (+  1455us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
1118 02:55:28.625541 (+203794us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50116: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"server-negotiator.queue_time_us":36839,"thread_start_us":43627,"threads_started":1}
W20261118 02:55:28.625998  3955 connection.cc:573] client connection to 127.2.229.125:38339 recv error: Network error: recv error from unknown peer: Connection reset by peer (error 104)
W20261118 02:55:28.626674  3955 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.229.125:38339)
W20261118 02:55:28.627084  3955 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.229.125:38339)
W20261118 02:55:28.627597  3955 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.2.229.125:38339)
W20261118 02:55:28.627985  3955 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.2.229.125:38339)
W20261118 02:55:28.628552  5631 negotiation.cc:336] Failed RPC negotiation. Trace:
1118 02:55:28.626985 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.2.229.125:38339 (local address 127.2.229.67:33901)
1118 02:55:28.627137 (+   152us) negotiation.cc:107] Waiting for socket to connect
1118 02:55:28.627153 (+    16us) client_negotiation.cc:175] Beginning negotiation
1118 02:55:28.627340 (+   187us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
1118 02:55:28.628508 (+  1168us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: BlockingRecv error: recv error from unknown peer: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":75}
I20261118 02:55:28.918025  5673 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:28.918331  5673 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:28.920075  5673 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:28.921134  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:28.922744  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:28.922981  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:28.923113  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:28.923457  5673 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:55:28.959977  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 1: this message will repeat every 5th retry.
W20261118 02:55:29.164395  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 1: this message will repeat every 5th retry.
I20261118 02:55:29.624663  5687 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:29.624853  5687 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:29.625605  5687 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:29.627727  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:29.627923  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:29.628039  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:29.628325  5687 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:55:30.372432  5701 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:30.372648  5701 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:30.373390  5701 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:30.375844  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:30.376075  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:30.376238  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:30.376500  5701 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:55:30.459797  2965 master-stress-test.cc:579] Killing test thread 1/25
W20261118 02:55:30.576287  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 6: this message will repeat every 5th retry.
W20261118 02:55:31.364048  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 6: this message will repeat every 5th retry.
I20261118 02:55:31.934599  5727 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:31.934803  5727 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:31.935712  5727 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:31.937803  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:31.938073  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:31.938205  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:31.938476  5727 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:55:32.381196  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 6: this message will repeat every 5th retry.
W20261118 02:55:32.388866  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 6: this message will repeat every 5th retry.
I20261118 02:55:33.085836  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:33.086050  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:33.448501  5750 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:33.448701  5750 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:33.449447  5750 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:33.451268  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:33.451478  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:33.451635  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:33.451882  5750 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:55:33.669234  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 6: this message will repeat every 5th retry.
I20261118 02:55:35.902992  5670 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:55:35.903170  5670 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:35.904042  5670 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:35.904940  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:55:35.905107  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:35.905292  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:35.905550  5670 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:36.518183  5784 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:36.518404  5784 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:36.519074  5784 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:36.519963  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20261118 02:55:36.521373  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:36.521543  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:36.521661  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:36.521893  5784 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:55:36.796990  5670 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:55:36.797199  5670 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:36.797950  5670 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:36.798799  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:55:36.798962  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:36.799101  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:36.799361  5670 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:38.087471  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:38.087716  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:55:39.108853  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 6: this message will repeat every 5th retry.
W20261118 02:55:39.387306  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 6: this message will repeat every 5th retry.
I20261118 02:55:40.308772  5833 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:40.308964  5833 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:40.309700  5833 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:40.311770  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:40.311973  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:40.312115  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:40.312369  5833 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:55:40.539942  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 11: this message will repeat every 5th retry.
W20261118 02:55:41.168836  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 11: this message will repeat every 5th retry.
W20261118 02:55:42.403877  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 11: this message will repeat every 5th retry.
W20261118 02:55:43.032873  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 11: this message will repeat every 5th retry.
I20261118 02:55:43.089282  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:43.089504  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:55:43.577225  5845 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:55:43.577451  5845 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:43.578172  5845 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:43.579113  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:55:43.579312  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:43.579429  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:43.579654  5845 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:55:44.005230  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 11: this message will repeat every 5th retry.
I20261118 02:55:45.397172  5885 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:45.397377  5885 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:45.398104  5885 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:45.399135  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 02:55:45.399145  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:45.400573  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:55:45.400753  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:55:45.400882  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:45.401109  5885 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:55:47.205823  5908 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:55:47.206045  5908 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:47.206779  5908 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:47.207985  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:55:47.208194  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:47.208325  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:47.208623  5908 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:48.091045  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:48.091236  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:55:48.731650  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 11: this message will repeat every 5th retry.
W20261118 02:55:49.752238  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 11: this message will repeat every 5th retry.
I20261118 02:55:50.034221  5932 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:55:50.034430  5932 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:50.035189  5932 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:50.036211  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:55:50.036546  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:50.036669  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:50.036922  5932 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:55:50.311463  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 16: this message will repeat every 5th retry.
W20261118 02:55:51.239516  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 16: this message will repeat every 5th retry.
W20261118 02:55:52.350610  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 16: this message will repeat every 5th retry.
W20261118 02:55:52.828927  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 16: this message will repeat every 5th retry.
I20261118 02:55:53.092849  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:53.093044  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:55:54.411811  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 16: this message will repeat every 5th retry.
I20261118 02:55:55.992439  5993 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:55:55.992622  5993 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:55:55.993381  5993 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:55:55.994601  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:55.996445  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:55:55.996683  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:55:55.996791  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:55:55.997062  5993 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:55:56.657282  5983 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:55:56.657473  5983 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:55:56.658321  5983 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:56.659269  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:55:56.659467  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:55:56.659564  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:56.659798  5983 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:57.104182  5983 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:55:57.104408  5983 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:55:57.105095  5983 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:55:57.106101  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:55:57.106492  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:55:57.106614  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:55:57.106855  5983 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:55:58.094657  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:55:58.094882  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:55:58.128293  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 16: this message will repeat every 5th retry.
W20261118 02:55:58.964799  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 16: this message will repeat every 5th retry.
W20261118 02:55:59.645067  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 21: this message will repeat every 5th retry.
W20261118 02:56:00.229477  3665 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (65 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:56:00.396555  3802 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (144 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:56:00.462844  3947 heartbeater.cc:648] Failed to heartbeat to 127.2.229.126:38025 (134 consecutive failures): Network error: Failed to ping master at 127.2.229.126:38025: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:56:00.605662  6045 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:56:00.605907  6045 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:56:00.606544  6045 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:00.607568  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:56:00.607733  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:56:00.607924  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:00.608206  6045 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:56:00.935271  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 21: this message will repeat every 5th retry.
W20261118 02:56:02.621012  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 21: this message will repeat every 5th retry.
W20261118 02:56:02.815258  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 21: this message will repeat every 5th retry.
I20261118 02:56:03.096472  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:03.096679  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:04.525568  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 21: this message will repeat every 5th retry.
I20261118 02:56:06.938094  6134 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:56:06.938328  6134 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:56:06.939091  6134 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:56:06.940845  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:56:06.942695  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:56:06.943400  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:56:06.943528  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:56:06.943857  6134 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:56:07.982607  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 21: this message will repeat every 5th retry.
I20261118 02:56:08.098405  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:08.098644  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:08.637887  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 21: this message will repeat every 5th retry.
W20261118 02:56:10.016618  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 26: this message will repeat every 5th retry.
W20261118 02:56:10.580835  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 26: this message will repeat every 5th retry.
W20261118 02:56:12.255952  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 26: this message will repeat every 5th retry.
I20261118 02:56:13.100533  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:13.100728  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:13.172420  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 26: this message will repeat every 5th retry.
W20261118 02:56:14.392789  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 26: this message will repeat every 5th retry.
I20261118 02:56:14.622538  6182 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:56:14.622720  6182 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:56:14.623581  6182 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:14.624326  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:56:14.624482  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:56:14.624579  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:14.624786  6182 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:56:16.350860  6236 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:56:16.351069  6236 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:56:16.351869  6236 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:56:16.352778  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:56:16.354365  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:56:16.354550  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:56:16.354700  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:56:16.354986  6236 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:56:18.102280  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:18.102491  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:18.447316  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 26: this message will repeat every 5th retry.
W20261118 02:56:18.752430  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 26: this message will repeat every 5th retry.
I20261118 02:56:19.769289  6182 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:56:19.769528  6182 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:56:19.770141  6182 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:19.771028  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:56:19.771209  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:56:19.771368  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:19.771590  6182 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:56:20.447530  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 31: this message will repeat every 5th retry.
W20261118 02:56:20.683594  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 31: this message will repeat every 5th retry.
W20261118 02:56:22.858696  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 31: this message will repeat every 5th retry.
I20261118 02:56:22.976730  6182 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:56:22.976922  6182 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:56:22.977869  6182 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:22.978744  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:56:22.978987  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:56:22.979118  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:22.979377  6182 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:56:23.104084  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:23.104324  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:56:23.131151  6182 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:56:23.131319  6182 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:56:23.131963  6182 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:23.132817  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:56:23.132997  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:56:23.133113  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:23.133387  6182 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:56:23.556182  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 31: this message will repeat every 5th retry.
W20261118 02:56:24.113736  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 31: this message will repeat every 5th retry.
W20261118 02:56:27.429958  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 31: this message will repeat every 5th retry.
I20261118 02:56:28.106338  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:28.106529  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:28.928059  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 31: this message will repeat every 5th retry.
I20261118 02:56:30.336418  6390 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:56:30.336602  6390 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:56:30.337323  6390 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:56:30.338326  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:56:30.339444  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:56:30.339612  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:56:30.339736  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:56:30.339988  6390 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:56:30.887456  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 36: this message will repeat every 5th retry.
W20261118 02:56:31.233016  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 36: this message will repeat every 5th retry.
W20261118 02:56:32.689002  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 36: this message will repeat every 5th retry.
W20261118 02:56:33.011636  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 36: this message will repeat every 5th retry.
I20261118 02:56:33.108178  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:33.108383  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:33.850236  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 36: this message will repeat every 5th retry.
W20261118 02:56:37.732273  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 36: this message will repeat every 5th retry.
I20261118 02:56:38.110023  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:38.110229  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:38.483652  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 36: this message will repeat every 5th retry.
W20261118 02:56:40.218343  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 41: this message will repeat every 5th retry.
W20261118 02:56:40.434991  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 41: this message will repeat every 5th retry.
I20261118 02:56:41.089998  6485 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:56:41.090183  6485 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:56:41.091089  6485 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:41.092043  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:56:41.092242  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:56:41.092366  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:41.092660  6485 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:56:42.392079  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 41: this message will repeat every 5th retry.
I20261118 02:56:42.624784  6506 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:56:42.625013  6506 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:56:42.625800  6506 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:42.626959  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:56:42.627652  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:56:42.627781  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:42.628055  6506 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:56:42.841060  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 41: this message will repeat every 5th retry.
I20261118 02:56:43.112290  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:43.112517  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:44.553699  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 41: this message will repeat every 5th retry.
I20261118 02:56:46.179849  6506 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:56:46.180191  6506 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:56:46.181200  6506 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:46.182098  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:56:46.182281  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:56:46.182389  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:46.182653  6506 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:56:48.114272  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:48.114487  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:48.187366  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 41: this message will repeat every 5th retry.
W20261118 02:56:48.359508  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 41: this message will repeat every 5th retry.
I20261118 02:56:49.149977  6592 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:56:49.150317  6592 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:56:49.151283  6592 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:56:49.152418  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:56:49.153699  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:56:49.153858  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:56:49.153991  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:56:49.154255  6592 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:56:50.503849  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 46: this message will repeat every 5th retry.
W20261118 02:56:50.921205  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 46: this message will repeat every 5th retry.
W20261118 02:56:51.940369  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 46: this message will repeat every 5th retry.
I20261118 02:56:53.115981  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:53.116219  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:53.304440  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 46: this message will repeat every 5th retry.
W20261118 02:56:54.531152  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 46: this message will repeat every 5th retry.
I20261118 02:56:56.163416  6506 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:56:56.163666  6506 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:56:56.164624  6506 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:56:56.165562  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:56:56.165759  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:56:56.165907  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:56:56.166239  6506 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:56:57.913360  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 46: this message will repeat every 5th retry.
I20261118 02:56:58.117758  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:56:58.117944  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:56:58.396592  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 46: this message will repeat every 5th retry.
I20261118 02:56:59.625134  6692 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:56:59.625365  6692 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:56:59.626210  6692 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:56:59.627149  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:56:59.628471  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:56:59.628677  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:56:59.628805  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:56:59.629104  6692 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:57:00.409845  6506 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:57:00.410043  6506 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:57:00.410719  6506 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:00.411444  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:57:00.411804  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:57:00.411923  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:00.412189  6506 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:57:00.416695  3665 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (185 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:57:00.675436  3947 heartbeater.cc:648] Failed to heartbeat to 127.2.229.126:38025 (361 consecutive failures): Network error: Failed to ping master at 127.2.229.126:38025: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:57:00.694525  3802 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (410 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:57:00.888291  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 51: this message will repeat every 5th retry.
W20261118 02:57:01.359127  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 51: this message will repeat every 5th retry.
W20261118 02:57:02.072229  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 51: this message will repeat every 5th retry.
I20261118 02:57:03.119546  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:03.119764  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:03.234062  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 51: this message will repeat every 5th retry.
W20261118 02:57:05.344251  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 51: this message will repeat every 5th retry.
I20261118 02:57:08.121307  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:08.121505  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:08.717660  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 51: this message will repeat every 5th retry.
W20261118 02:57:09.268417  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 51: this message will repeat every 5th retry.
I20261118 02:57:09.348333  6794 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:57:09.348572  6794 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:57:09.349795  6794 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:57:09.350687  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:57:09.351788  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:57:09.351953  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:57:09.352061  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:57:09.352324  6794 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:57:10.313709  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 56: this message will repeat every 5th retry.
W20261118 02:57:11.008773  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 56: this message will repeat every 5th retry.
W20261118 02:57:12.083596  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 56: this message will repeat every 5th retry.
W20261118 02:57:12.877405  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 56: this message will repeat every 5th retry.
I20261118 02:57:13.122869  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:13.123042  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:57:13.527782  6844 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:57:13.527964  6844 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:57:13.528836  6844 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:13.529757  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:57:13.529932  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:57:13.530035  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:13.530251  6844 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:57:13.782737  6844 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:57:13.782931  6844 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:57:13.783641  6844 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:13.784574  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:57:13.784770  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:57:13.784873  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:13.785085  6844 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:57:15.527200  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 56: this message will repeat every 5th retry.
I20261118 02:57:18.124677  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:18.124908  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:18.888922  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 56: this message will repeat every 5th retry.
W20261118 02:57:19.224823  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 61: this message will repeat every 5th retry.
W20261118 02:57:19.643155  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 56: this message will repeat every 5th retry.
I20261118 02:57:21.382807  6935 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:57:21.383019  6935 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:57:21.383836  6935 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:57:21.385071  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:57:21.386317  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:57:21.386483  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:57:21.386593  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:57:21.386862  6935 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:57:21.638518  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 61: this message will repeat every 5th retry.
I20261118 02:57:22.537374  6844 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:57:22.537598  6844 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:57:22.538316  6844 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:22.539288  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:57:22.539484  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:57:22.539587  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:22.539827  6844 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:57:22.608337  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 61: this message will repeat every 5th retry.
I20261118 02:57:23.059811  6958 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:57:23.060045  6958 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:57:23.060866  6958 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:57:23.062950  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:57:23.063333  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:57:23.063444  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:57:23.063720  6958 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:57:23.126088  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:23.126300  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:23.227061  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 61: this message will repeat every 5th retry.
W20261118 02:57:25.723588  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 61: this message will repeat every 5th retry.
I20261118 02:57:27.518970  6996 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:57:27.519127  6996 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:57:27.519719  6996 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:27.520565  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:57:27.520710  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:57:27.520828  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:27.521050  6996 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:57:28.086007  5559 maintenance_manager.cc:419] P 813a6937cea442ef81645fccc32207f4: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033343
I20261118 02:57:28.127781  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:28.127975  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:57:28.133642  5558 maintenance_manager.cc:643] P 813a6937cea442ef81645fccc32207f4: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.047s	user 0.025s	sys 0.007s Metrics: {"bytes_written":27552,"cfile_init":1,"dirs.queue_time_us":395,"dirs.run_cpu_time_us":326,"dirs.run_wall_time_us":1120,"drs_written":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4,"lbm_write_time_us":691,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":74,"thread_start_us":853,"threads_started":2,"wal-append.queue_time_us":680}
I20261118 02:57:28.134949  5559 maintenance_manager.cc:419] P 813a6937cea442ef81645fccc32207f4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4582 bytes on disk
I20261118 02:57:28.135690  5558 maintenance_manager.cc:643] P 813a6937cea442ef81645fccc32207f4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4}
I20261118 02:57:28.257326  6996 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:57:28.257539  6996 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:57:28.258230  6996 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:28.259096  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:57:28.259222  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:57:28.259339  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:28.259589  6996 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:57:29.153463  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 61: this message will repeat every 5th retry.
W20261118 02:57:29.218022  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 66: this message will repeat every 5th retry.
W20261118 02:57:29.278539  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 61: this message will repeat every 5th retry.
W20261118 02:57:32.317335  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 66: this message will repeat every 5th retry.
W20261118 02:57:33.106753  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 66: this message will repeat every 5th retry.
I20261118 02:57:33.129535  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:33.129786  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:33.193670  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 66: this message will repeat every 5th retry.
W20261118 02:57:35.607445  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 66: this message will repeat every 5th retry.
I20261118 02:57:38.131350  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:38.131604  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:38.915306  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 71: this message will repeat every 5th retry.
W20261118 02:57:39.266597  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 66: this message will repeat every 5th retry.
W20261118 02:57:39.697613  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 66: this message will repeat every 5th retry.
I20261118 02:57:40.594195  7202 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:57:40.594416  7202 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:57:40.595391  7202 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:57:40.596397  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 02:57:40.598003  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:57:40.598182  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:57:40.598346  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:57:40.598616  7202 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:57:42.236785  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 71: this message will repeat every 5th retry.
W20261118 02:57:42.904325  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 71: this message will repeat every 5th retry.
I20261118 02:57:43.133165  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:43.133348  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:43.382508  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 71: this message will repeat every 5th retry.
I20261118 02:57:44.654345  7105 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:57:44.654580  7105 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:57:44.655544  7105 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:44.656658  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:57:44.656867  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:57:44.656953  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:44.657202  7105 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:57:45.286657  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 71: this message will repeat every 5th retry.
I20261118 02:57:47.943230  7105 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:57:47.943408  7105 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:57:47.944044  7105 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:47.945016  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:57:47.945175  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:57:47.945299  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:47.945525  7304 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:57:48.135010  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:48.135201  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:48.875918  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 76: this message will repeat every 5th retry.
W20261118 02:57:49.223453  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 71: this message will repeat every 5th retry.
W20261118 02:57:49.766619  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 71: this message will repeat every 5th retry.
I20261118 02:57:52.349977  7360 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:57:52.350210  7360 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:57:52.350996  7360 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:57:52.352134  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:57:52.353381  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:57:52.353543  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:57:52.353725  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:57:52.354013  7360 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:57:52.404729  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 76: this message will repeat every 5th retry.
I20261118 02:57:53.136974  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:53.137172  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:53.254227  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 76: this message will repeat every 5th retry.
W20261118 02:57:53.552588  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 76: this message will repeat every 5th retry.
W20261118 02:57:54.932454  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 76: this message will repeat every 5th retry.
I20261118 02:57:58.138635  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:57:58.138833  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:57:59.018529  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 81: this message will repeat every 5th retry.
I20261118 02:57:59.559317  7419 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:57:59.559527  7419 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:57:59.560314  7419 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:57:59.561225  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:57:59.561384  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:57:59.561528  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:57:59.561766  7419 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:57:59.565672  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 76: this message will repeat every 5th retry.
W20261118 02:58:00.144567  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 76: this message will repeat every 5th retry.
W20261118 02:58:00.616329  3665 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (305 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:58:00.958791  3949 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (571 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:58:00.973905  3802 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (683 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:58:02.822342  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 81: this message will repeat every 5th retry.
I20261118 02:58:02.847697  7482 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:58:02.847905  7482 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:58:02.848784  7482 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:02.849704  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:58:02.849859  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:58:02.849999  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:02.850291  7482 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:58:02.888734  7482 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:58:02.888943  7482 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:58:02.889647  7482 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:02.890311  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:58:02.890470  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:58:02.890573  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:02.890769  7482 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:58:03.050968  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 81: this message will repeat every 5th retry.
I20261118 02:58:03.140501  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:03.140720  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:03.245304  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 81: this message will repeat every 5th retry.
I20261118 02:58:05.032606  7516 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:58:05.032799  7516 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:58:05.033535  7516 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:58:05.034482  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:58:05.035732  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:58:05.035881  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:58:05.036039  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:58:05.036317  7516 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:58:05.130908  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 81: this message will repeat every 5th retry.
I20261118 02:58:06.861006  7545 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:58:06.861192  7545 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:58:06.861899  7545 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:06.862658  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:58:06.862838  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:58:06.862974  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:06.863242  7545 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:58:08.142208  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:08.142498  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:08.980360  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 81: this message will repeat every 5th retry.
W20261118 02:58:08.997622  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 86: this message will repeat every 5th retry.
W20261118 02:58:10.549921  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 81: this message will repeat every 5th retry.
W20261118 02:58:12.724419  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 86: this message will repeat every 5th retry.
I20261118 02:58:13.144183  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:13.144383  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:13.223852  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 86: this message will repeat every 5th retry.
W20261118 02:58:13.358853  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 86: this message will repeat every 5th retry.
I20261118 02:58:13.973117  7545 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:58:13.973318  7545 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:58:13.973928  7545 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:13.974880  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:58:13.975062  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:58:13.975190  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:13.975451  7545 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:58:15.452023  7660 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:58:15.452253  7660 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:58:15.453040  7660 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:58:15.454002  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:58:15.455380  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:58:15.455614  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:58:15.455732  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:58:15.456010  7660 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:58:15.732462  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 86: this message will repeat every 5th retry.
I20261118 02:58:18.146245  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:18.146437  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:18.375077  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 91: this message will repeat every 5th retry.
W20261118 02:58:19.783526  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 86: this message will repeat every 5th retry.
W20261118 02:58:20.572118  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 86: this message will repeat every 5th retry.
W20261118 02:58:22.920607  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 91: this message will repeat every 5th retry.
W20261118 02:58:23.047988  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 91: this message will repeat every 5th retry.
I20261118 02:58:23.148067  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:23.148340  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:23.243335  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 91: this message will repeat every 5th retry.
I20261118 02:58:23.716390  7776 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:58:23.716588  7776 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:58:23.717418  7776 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:58:23.718448  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:58:23.719712  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:58:23.719890  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:58:23.719997  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:58:23.720254  7776 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:58:24.423086  7715 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:58:24.423274  7715 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:58:24.424043  7715 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:24.424883  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:58:24.425033  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:58:24.425149  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:24.425343  7715 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:58:25.530254  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 91: this message will repeat every 5th retry.
I20261118 02:58:28.150002  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:28.150254  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:28.207217  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 96: this message will repeat every 5th retry.
I20261118 02:58:28.532006  7715 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:58:28.532238  7715 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:58:28.532825  7715 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:28.533812  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:58:28.533962  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:58:28.534099  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:28.534385  7715 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:58:29.613914  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 91: this message will repeat every 5th retry.
W20261118 02:58:30.316879  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 91: this message will repeat every 5th retry.
W20261118 02:58:32.397710  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 96: this message will repeat every 5th retry.
W20261118 02:58:32.811247  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 96: this message will repeat every 5th retry.
I20261118 02:58:33.151935  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:33.152170  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:33.206007  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 96: this message will repeat every 5th retry.
W20261118 02:58:35.088420  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 96: this message will repeat every 5th retry.
I20261118 02:58:35.644779  7917 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:58:35.644985  7917 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:58:35.645798  7917 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:58:35.646842  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:58:35.648344  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:58:35.648530  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:58:35.648663  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:58:35.648937  7917 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:58:36.542936  7851 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:58:36.543102  7851 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:58:36.543743  7851 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:36.544659  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:58:36.544800  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:58:36.544909  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:36.545130  7851 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:58:38.025601  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 101: this message will repeat every 5th retry.
I20261118 02:58:38.153779  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:38.154428  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:58:39.628973  7943 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:58:39.629137  7943 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:58:39.630405  7943 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:39.631377  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:58:39.631531  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:58:39.631620  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:39.631844  7943 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:58:39.912817  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 96: this message will repeat every 5th retry.
W20261118 02:58:40.110134  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 96: this message will repeat every 5th retry.
I20261118 02:58:40.765530  7990 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:58:40.765715  7990 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:58:40.766443  7990 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:58:40.767242  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:58:40.768546  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:58:40.768729  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:58:40.768850  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:58:40.769096  7990 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:58:42.661867  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 101: this message will repeat every 5th retry.
I20261118 02:58:43.156122  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:43.156363  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:43.337124  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 101: this message will repeat every 5th retry.
W20261118 02:58:43.376363  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 101: this message will repeat every 5th retry.
I20261118 02:58:43.912972  7943 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:58:43.913167  7943 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:58:43.913892  7943 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:43.914738  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:58:43.914909  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:58:43.915009  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:43.915249  7943 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:58:44.596998  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 101: this message will repeat every 5th retry.
W20261118 02:58:47.813879  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 106: this message will repeat every 5th retry.
I20261118 02:58:48.119235  8080 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:58:48.119484  8080 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:58:48.120294  8080 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:58:48.121352  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:58:48.122648  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:58:48.122849  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:58:48.122973  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:58:48.123236  8080 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:58:48.157598  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:48.157836  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:49.309739  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 101: this message will repeat every 5th retry.
W20261118 02:58:49.761521  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 101: this message will repeat every 5th retry.
I20261118 02:58:50.941085  7943 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:58:50.941250  7943 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:58:50.941870  7943 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:50.942737  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:58:50.942862  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:58:50.942963  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:50.943217  7943 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:58:52.901240  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 106: this message will repeat every 5th retry.
I20261118 02:58:53.159374  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:53.159590  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:53.184293  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 106: this message will repeat every 5th retry.
W20261118 02:58:53.250200  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 106: this message will repeat every 5th retry.
W20261118 02:58:54.610394  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 106: this message will repeat every 5th retry.
I20261118 02:58:55.165832  8157 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:58:55.166010  8157 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:58:55.166731  8157 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:58:55.167569  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:58:55.168864  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:58:55.169199  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:58:55.169319  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:58:55.169595  8157 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:58:58.157732  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 111: this message will repeat every 5th retry.
I20261118 02:58:58.160801  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:58:58.160995  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:58:59.485185  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 106: this message will repeat every 5th retry.
I20261118 02:58:59.602751  8182 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:58:59.603106  8182 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:58:59.604077  8182 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:58:59.604846  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:58:59.604982  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:58:59.605069  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:58:59.605238  8182 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:59:00.474730  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 106: this message will repeat every 5th retry.
W20261118 02:59:00.793666  3665 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (425 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:59:01.046530  3949 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (794 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:59:01.386315  3802 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (951 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:59:02.635401  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 111: this message will repeat every 5th retry.
W20261118 02:59:02.814745  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 111: this message will repeat every 5th retry.
I20261118 02:59:03.162468  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:03.162700  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 02:59:03.315689  8182 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:59:03.315883  8182 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:59:03.316619  8182 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:03.317687  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:59:03.318092  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:59:03.318188  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:03.318475  8182 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:59:04.083133  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 111: this message will repeat every 5th retry.
W20261118 02:59:04.494549  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 111: this message will repeat every 5th retry.
I20261118 02:59:04.678925  8253 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:59:04.679127  8253 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:59:04.679788  8253 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:04.680799  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:59:04.681092  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:59:04.681190  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:04.681406  8253 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:59:06.879763  8277 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:06.879956  8277 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:06.880702  8277 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:06.881628  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 02:59:06.882915  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:59:06.883103  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:59:06.883216  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:06.883445  8277 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:59:07.705319  8253 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:59:07.705505  8253 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:59:07.706146  8253 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:07.707072  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:59:07.707245  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:59:07.707355  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:07.707620  8253 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:59:07.984573  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 116: this message will repeat every 5th retry.
I20261118 02:59:08.164357  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:08.164602  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:09.783567  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 111: this message will repeat every 5th retry.
W20261118 02:59:10.100283  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 111: this message will repeat every 5th retry.
I20261118 02:59:10.226858  8316 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:10.227104  8316 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:10.227859  8316 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:10.230258  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:59:10.230464  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:59:10.230607  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:10.230877  8316 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:59:12.440300  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 116: this message will repeat every 5th retry.
W20261118 02:59:12.704334  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 116: this message will repeat every 5th retry.
I20261118 02:59:12.794190  8350 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:12.794389  8350 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:12.795086  8350 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:12.796036  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 02:59:12.797224  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:59:12.797408  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:59:12.797526  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:12.797791  8350 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:59:13.165732  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:13.165930  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:13.383304  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 116: this message will repeat every 5th retry.
W20261118 02:59:13.880877  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 116: this message will repeat every 5th retry.
I20261118 02:59:14.775440  8373 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:14.775702  8373 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:14.776453  8373 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:14.778760  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:59:14.778945  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:59:14.779065  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:14.779287  8373 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:59:17.219662  8253 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:59:17.219847  8253 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:59:17.220664  8253 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:17.221486  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:59:17.221627  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:59:17.221725  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:17.221954  8253 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:59:18.167467  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:18.167695  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:18.357481  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 121: this message will repeat every 5th retry.
I20261118 02:59:18.581887  8253 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:59:18.582090  8253 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:59:18.582803  8253 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:18.583911  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:59:18.584656  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:59:18.584745  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:18.584959  8253 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:59:20.007988  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 116: this message will repeat every 5th retry.
W20261118 02:59:20.645378  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 116: this message will repeat every 5th retry.
W20261118 02:59:22.600922  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 121: this message will repeat every 5th retry.
W20261118 02:59:23.137544  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 121: this message will repeat every 5th retry.
I20261118 02:59:23.169693  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:23.169919  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:23.474047  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 121: this message will repeat every 5th retry.
W20261118 02:59:24.151412  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 121: this message will repeat every 5th retry.
I20261118 02:59:27.873066  8496 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:59:27.873217  8496 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:59:27.873821  8496 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:27.874632  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:59:27.874763  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:59:27.874897  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:27.875106  8496 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:59:28.171466  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:28.171648  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:28.240993  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 126: this message will repeat every 5th retry.
W20261118 02:59:29.834745  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 121: this message will repeat every 5th retry.
W20261118 02:59:30.827389  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 121: this message will repeat every 5th retry.
I20261118 02:59:31.229068  8496 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:59:31.229236  8496 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:59:31.229826  8496 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:31.230625  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:59:31.230741  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:59:31.230862  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:31.231081  8496 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:59:32.790480  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 126: this message will repeat every 5th retry.
I20261118 02:59:32.855837  8561 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:32.856045  8561 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:32.856999  8561 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:32.858637  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 02:59:32.859872  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:59:32.860028  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:59:32.860193  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:32.860458  8561 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 02:59:32.952739  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 126: this message will repeat every 5th retry.
I20261118 02:59:33.172973  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:33.173195  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:33.334769  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 126: this message will repeat every 5th retry.
W20261118 02:59:33.904570  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 126: this message will repeat every 5th retry.
I20261118 02:59:35.417480  8582 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:35.417649  8582 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:35.418402  8582 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:35.421079  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:59:35.421217  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:59:35.421314  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:35.421484  8582 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:59:36.381064  8567 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:59:36.381253  8567 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:59:36.382048  8567 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:36.382725  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:59:36.382855  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:59:36.382943  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:36.383144  8567 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 02:59:38.174733  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:38.174988  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:38.235090  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 131: this message will repeat every 5th retry.
W20261118 02:59:39.591650  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 126: this message will repeat every 5th retry.
W20261118 02:59:40.303997  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 126: this message will repeat every 5th retry.
I20261118 02:59:41.319796  8567 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:59:41.320008  8567 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:59:41.320714  8567 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:41.321551  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:59:41.321691  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:59:41.321800  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:41.322021  8567 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:59:43.006160  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 131: this message will repeat every 5th retry.
W20261118 02:59:43.125178  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 131: this message will repeat every 5th retry.
W20261118 02:59:43.166941  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 131: this message will repeat every 5th retry.
I20261118 02:59:43.176337  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:43.176496  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:43.211259  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 131: this message will repeat every 5th retry.
I20261118 02:59:43.725240  8663 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:43.725435  8663 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:43.726222  8663 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:43.727092  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 02:59:43.728869  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:59:43.729112  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:59:43.729187  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:43.729442  8663 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:59:47.651849  8708 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:47.652037  8708 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:47.652654  8708 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:47.654553  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 02:59:47.654690  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 02:59:47.654790  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:47.655011  8708 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:59:48.177984  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:48.178169  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:48.391852  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 136: this message will repeat every 5th retry.
W20261118 02:59:49.471623  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 131: this message will repeat every 5th retry.
W20261118 02:59:50.582154  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 131: this message will repeat every 5th retry.
I20261118 02:59:51.978672  8728 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 02:59:51.978927  8728 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 02:59:51.979714  8728 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:51.980540  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 02:59:51.980695  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 02:59:51.980810  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:51.980988  8728 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:59:52.198585  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 136: this message will repeat every 5th retry.
I20261118 02:59:52.921782  8728 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 02:59:52.921998  8728 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 02:59:52.922627  8728 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 02:59:52.923377  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 02:59:52.923636  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 02:59:52.923750  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 02:59:52.923982  8728 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 02:59:53.053478  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 136: this message will repeat every 5th retry.
I20261118 02:59:53.179688  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:53.179870  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:53.523923  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 136: this message will repeat every 5th retry.
W20261118 02:59:53.575775  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 136: this message will repeat every 5th retry.
I20261118 02:59:57.373075  8794 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 02:59:57.373271  8794 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 02:59:57.374028  8794 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 02:59:57.375049  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20261118 02:59:57.376375  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 02:59:57.376556  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 02:59:57.376715  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 02:59:57.376955  8794 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 02:59:58.181721  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 02:59:58.181943  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 02:59:58.394097  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 141: this message will repeat every 5th retry.
W20261118 02:59:59.626570  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 136: this message will repeat every 5th retry.
W20261118 03:00:00.265889  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 136: this message will repeat every 5th retry.
W20261118 03:00:00.933529  3665 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (545 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 03:00:01.445329  3949 heartbeater.cc:648] Failed to heartbeat to 127.2.229.125:38339 (1022 consecutive failures): Network error: Failed to ping master at 127.2.229.125:38339: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 03:00:01.534240  3800 heartbeater.cc:648] Failed to heartbeat to 127.2.229.126:38025 (1244 consecutive failures): Network error: Failed to ping master at 127.2.229.126:38025: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 03:00:01.534797  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 141: this message will repeat every 5th retry.
I20261118 03:00:03.183622  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 03:00:03.183862  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 03:00:03.427086  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 141: this message will repeat every 5th retry.
I20261118 03:00:03.739813  8869 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 03:00:03.740005  8869 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 03:00:03.740705  8869 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 03:00:03.741703  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 03:00:03.743103  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
W20261118 03:00:03.743252  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
I20261118 03:00:03.743361  5555 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 03:00:03.743626  8869 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 03:00:03.811874  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 141: this message will repeat every 5th retry.
W20261118 03:00:03.989079  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 141: this message will repeat every 5th retry.
I20261118 03:00:04.976514  8833 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 03:00:04.976691  8833 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 03:00:04.977308  8833 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 03:00:04.978266  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 03:00:04.978451  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 03:00:04.978546  3552 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 03:00:04.978761  8833 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 03:00:07.900988  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 146: this message will repeat every 5th retry.
I20261118 03:00:08.185303  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 03:00:08.185462  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 03:00:10.116611  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 141: this message will repeat every 5th retry.
I20261118 03:00:10.539739  8910 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 03:00:10.539944  8910 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 03:00:10.540849  8910 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 03:00:10.541875  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 03:00:10.542029  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 03:00:10.542129  3553 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 03:00:10.542333  8910 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 03:00:10.730866  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 141: this message will repeat every 5th retry.
W20261118 03:00:11.152266  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 146: this message will repeat every 5th retry.
I20261118 03:00:11.578753  8938 raft_consensus.cc:499] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 99a39b1d9d814deeb3a8a644e9b392ef)
I20261118 03:00:11.578953  8938 raft_consensus.cc:521] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } } peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } }
I20261118 03:00:11.579612  8938 leader_election.cc:290] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 03:00:11.580538  3552 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
W20261118 03:00:11.585196  3553 leader_election.cc:343] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: a013d180f2cd430ab1ba30b39fd27591
I20261118 03:00:11.585372  3553 leader_election.cc:304] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 03:00:11.585675  8938 raft_consensus.cc:2771] T a013d180f2cd430ab1ba30b39fd27591 P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 03:00:13.009191  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 146: this message will repeat every 5th retry.
I20261118 03:00:13.187103  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 03:00:13.187258  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 03:00:13.862955  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 146: this message will repeat every 5th retry.
W20261118 03:00:14.415006  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 146: this message will repeat every 5th retry.
W20261118 03:00:17.568670  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 151: this message will repeat every 5th retry.
I20261118 03:00:18.188782  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 03:00:18.188958  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20261118 03:00:18.405079  8938 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 03:00:18.405267  8938 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 03:00:18.405918  8938 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 03:00:18.406755  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 03:00:18.406901  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 03:00:18.407008  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 03:00:18.407233  8938 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20261118 03:00:20.692251  3838 consensus_peers.cc:597] T 2b01d7e596bd4790b8ba5dc88d8e12cc P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b01d7e596bd4790b8ba5dc88d8e12cc. This is attempt 146: this message will repeat every 5th retry.
W20261118 03:00:20.773861  3691 consensus_peers.cc:597] T 8ae679d2feb04b95ab2f4fb51e443d64 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ae679d2feb04b95ab2f4fb51e443d64. This is attempt 151: this message will repeat every 5th retry.
W20261118 03:00:20.833590  3691 consensus_peers.cc:597] T 0e901a03b90e4c978eb98be87163cfe5 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e901a03b90e4c978eb98be87163cfe5. This is attempt 146: this message will repeat every 5th retry.
I20261118 03:00:22.821983  9055 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 03:00:22.822163  9055 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 03:00:22.823007  9055 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 03:00:22.824005  5556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 03:00:22.825198  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 03:00:22.825367  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 03:00:22.825480  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 03:00:22.825721  9055 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20261118 03:00:23.190071  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 03:00:23.190238  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 03:00:23.469233  3691 consensus_peers.cc:597] T ffe8d4c111d14054b76f9c7724072a94 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffe8d4c111d14054b76f9c7724072a94. This is attempt 151: this message will repeat every 5th retry.
W20261118 03:00:24.134795  3838 consensus_peers.cc:597] T 973831392ec24e0681c3d4fffd8bb199 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 973831392ec24e0681c3d4fffd8bb199. This is attempt 151: this message will repeat every 5th retry.
W20261118 03:00:24.519721  3838 consensus_peers.cc:597] T c0713dcf79f54873a99d7556ad8f1454 P 3f1973986ec1466eab5bec01dd81654f -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0713dcf79f54873a99d7556ad8f1454. This is attempt 151: this message will repeat every 5th retry.
I20261118 03:00:24.894520  8938 raft_consensus.cc:499] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3f1973986ec1466eab5bec01dd81654f)
I20261118 03:00:24.894719  8938 raft_consensus.cc:521] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fdf7d7ece324fefb3a0c376add5b7e8" member_type: VOTER last_known_addr { host: "127.2.229.65" port: 33705 } } peers { permanent_uuid: "99a39b1d9d814deeb3a8a644e9b392ef" member_type: VOTER last_known_addr { host: "127.2.229.66" port: 40377 } } peers { permanent_uuid: "3f1973986ec1466eab5bec01dd81654f" member_type: VOTER last_known_addr { host: "127.2.229.67" port: 42237 } }
I20261118 03:00:24.895377  8938 leader_election.cc:290] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377), 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237)
W20261118 03:00:24.896324  3553 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f1973986ec1466eab5bec01dd81654f (127.2.229.67:42237): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
W20261118 03:00:24.896512  3552 leader_election.cc:343] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 99a39b1d9d814deeb3a8a644e9b392ef (127.2.229.66:40377): Not found: Tablet not found: c92c29c30ab6411dabe6f02331d29b7a
I20261118 03:00:24.896611  3552 leader_election.cc:304] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fdf7d7ece324fefb3a0c376add5b7e8; no voters: 3f1973986ec1466eab5bec01dd81654f, 99a39b1d9d814deeb3a8a644e9b392ef
I20261118 03:00:24.896838  8938 raft_consensus.cc:2771] T c92c29c30ab6411dabe6f02331d29b7a P 1fdf7d7ece324fefb3a0c376add5b7e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20261118 03:00:28.191579  5526 master_runner.cc:187] Error getting master registration for 127.2.229.126:38025: Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111), 
I20261118 03:00:28.191725  5526 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20261118 03:00:28.381434  3691 consensus_peers.cc:597] T b82ca8a282f94392bb2746ed7b8b8ba7 P 99a39b1d9d814deeb3a8a644e9b392ef -> Peer 1fdf7d7ece324fefb3a0c376add5b7e8 (127.2.229.65:33705): Couldn't send request to peer 1fdf7d7ece324fefb3a0c376add5b7e8. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b82ca8a282f94392bb2746ed7b8b8ba7. This is attempt 156: this message will repeat every 5th retry.
I20261118 03:00:28.420482  5559 maintenance_manager.cc:419] P 813a6937cea442ef81645fccc32207f4: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4582 bytes on disk
I20261118 03:00:28.425880  5558 maintenance_manager.cc:643] P 813a6937cea442ef81645fccc32207f4: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s	user 0.004s	sys 0.000s Metrics: {"dirs.queue_time_us":332,"dirs.run_cpu_time_us":89,"dirs.run_wall_time_us":90,"thread_start_us":480,"threads_started":2,"wal-append.queue_time_us":333}
I20261118 03:00:28.504838  9149 raft_consensus.cc:499] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 457eb38cba564ecd9f6f11245172260b)
I20261118 03:00:28.505012  9149 raft_consensus.cc:521] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c6b20fa203404a88463ea27bdd3c7d" member_type: VOTER last_known_addr { host: "127.2.229.126" port: 38025 } } peers { permanent_uuid: "457eb38cba564ecd9f6f11245172260b" member_type: VOTER last_known_addr { host: "127.2.229.125" port: 38339 } } peers { permanent_uuid: "813a6937cea442ef81645fccc32207f4" member_type: VOTER last_known_addr { host: "127.2.229.124" port: 36695 } }
I20261118 03:00:28.505633  9149 leader_election.cc:290] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025), 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339)
W20261118 03:00:28.506227  5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20261118 03:00:28.507370  5555 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 457eb38cba564ecd9f6f11245172260b (127.2.229.125:38339): Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 03:00:28.507586  5556 leader_election.cc:336] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 26c6b20fa203404a88463ea27bdd3c7d (127.2.229.126:38025): Network error: Client connection negotiation failed: client connection to 127.2.229.126:38025: connect: Connection refused (error 111)
I20261118 03:00:28.507695  5556 leader_election.cc:304] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 813a6937cea442ef81645fccc32207f4; no voters: 26c6b20fa203404a88463ea27bdd3c7d, 457eb38cba564ecd9f6f11245172260b
I20261118 03:00:28.507944  9149 raft_consensus.cc:2771] T 00000000000000000000000000000000 P 813a6937cea442ef81645fccc32207f4 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20261118 03:00:28.633126  3956 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 03:00:28.633379  3956 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 03:00:28.633582  3956 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
W20261118 03:00:28.633705  3956 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
F20261118 03:00:28.644670  3968 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_52fca7a2e65f4e679bbef9de3df569f1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Aborted at 1794970828 (unix time) try "date -d @1794970828" if you are using GNU date ***
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: T*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650763  3988 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e800000b95) received by PID 2965 (TID 0x7f8796674700) from PID 2965; stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650763  3988 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.651935  3978 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
    @     0x7f87ac1f4980 (unknown) at ??:0
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650763  3988 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.651935  3978 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653137  3976 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653162  3973 master-stress-test.cc:330] Check failed: _s.ok() Bad status: T*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650763  3988 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.651935  3978 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653137  3976 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653162  3973 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650763  3988 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.651935  3978 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653137  3976 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653162  3973 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653373  3981 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650763  3988 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.651935  3978 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653137  3976 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653162  3973 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653373  3981 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653424  3967 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e39f59e2fdbb47fda4c10beba29ad8f1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650763  3988 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.651935  3978 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653137  3976 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653162  3973 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653373  3981 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653424  3967 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e39f59e2fdbb47fda4c10beba29ad8f1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653592  3974 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
F20261118 03:00:28.644738  3979 master-stress-test.cc:353] Check failed: _s.ok() Bad status: F20261118 03:00:28.644739 Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.644862  3972 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646270  3964 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_51d36ff687ca47b5995e5693d7b8931b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646327  3982 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.646515  3966 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_bdbed231860946c2bf9b514f724bc3e9 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648559  3975 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648626  3986 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.648727  3987 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.649066  3962 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_dc982811b980416484416db1f74c685d on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650036  3977 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650137  3969 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_1d3bb660e14d4101996960be8caa8f81 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650185  3965 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_fd4fcac1f1d946fa898176f1d5e87e9b on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3983 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650679  3963 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_8139578347f041afbe51671d3be78447 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650697  3971 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_ee3815d9524243ec886bd171ccaff9e2 on the master: CreateTable timed out after deadline expired: CreateTable passed its     @     0x7f87a4f64e87 gsignal
deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.650763  3988 master-stress-test.cc:393] Check failed: _s.ok() Bad status: Timed out: ReplaceTablet timed out after deadline expired: ReplaceTablet passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.651935  3978 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653137  3976 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653162  3973 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653373  3981 master-stress-test.cc:353] Check failed: _s.ok() Bad status: Timed out: DeleteTable timed out after deadline expired: DeleteTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653424  3967 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_e39f59e2fdbb47fda4c10beba29ad8f1 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653592  3974 master-stress-test.cc:330] Check failed: _s.ok() Bad status: Timed out: AlterTable timed out after deadline expired: AlterTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)F20261118 03:00:28.653714  3970 master-stress-test.cc:299] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_40868189cecf45d0b705461c4328ee17 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.229.126:38025,127.2.229.125:38339,127.2.229.124:36695, num_attempts: 769) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.229.125:38339: connect: Connection refused (error 111)
*** Check failure stack trace: ***
    @     0x7f87a4f667f1 abort at ??:0
    @     0x7f87a7cb3d5d google::LogMessage::Fail() at ??:0
    @     0x7f87a7cb7b23 google::LogMessage::SendToLog() at ??:0
    @     0x7f87a7cb375c google::LogMessage::Flush() at ??:0
    @     0x7f87a7cb4ee9 google::LogMessageFatal::~LogMessageFatal() at ??:0
    @           0x409062 kudu::MasterStressTest::CreateTableThread() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/master-stress-test.cc:299
    @           0x447484 std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73
    @           0x44733d std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
    @           0x4472fa _ZNSt6thread8_InvokerISt5tupleIJMN4kudu16MasterStressTestEFvvEPNS2_26MasterStressTest_Test_TestEEEE9_M_invokeIJLm0ELm1EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
    @           0x4472ae std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
    @           0x446fe9 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
    @     0x7f87a55ee6df (unknown) at ??:0
    @     0x7f87ac1e96db start_thread at ??:0
    @     0x7f87a504761f clone at ??:0